Development environment guide with Intellij and Docker-Compose

[Hanelore]

Please follow these steps to setup your development environment using IntelliJ IDEA and Docker.
Let us know in case you need additional help.

This guide uses Intellij IDEA 2020.3. Always use the latest Idea version available. There may be slight configuration differences between idea versions, but we believe you can figure things out .

This guide is intended for Windows developers and uses powershell and not cmd for any CLI examples.

My windows account username is crist so whenever you see that, please replace it with yours.

Install git

Download and install git from https://git-scm.com/downloads, then make sure it is added to your PATH.

To check if it’s added to PATH open powershell and run:

git --version

You should get an output similar to this:

PS C:\Users\crist> git --version
git version 2.22.0.windows.1

Clone metasfresh project

All the metasfresh projects will be located in

C:/work-metas

this is the root folder of our project.

The following code will create work-metas then clone all the metasfresh modules needed from github:

mkdir c:/work-metas
cd C:/work-metas

git clone https://github.com/metasfresh/metasfresh

Install OpenJDK Java 8

Download and install JDK 8 from AdoptOpenJDK, then set the JAVA_HOME path variable.

To add the path variable using powershell with administrator rights, run:

setx JAVA_HOME 'C:\Program Files\Java\jdk1.8.0_212'

SUCCESS: Specified value was saved.

Install Node.js

Download and install the latest LTS version of Node.js. This is needed for the React metasfresh-webui-frontend and for metasfresh-e2e projects.

Install Postgres Command Line Tools

There’s no need to install the full Postgres database. Metasfresh uses psql for running the sql migrations, and it is found in the Command Line Tools.

Download the installer for postgresql version 10 (not a later version) from https://www.enterprisedb.com/downloads/postgres-postgresql-downloads, and select Command Line Tools:

then add the bin folder to Path with

$PATH = [Environment]::GetEnvironmentVariable("PATH", "User")
$pg_path = "C:\Program Files\PostgreSQL\10\bin"
[Environment]::SetEnvironmentVariable("PATH", "$PATH;$pg_path", "User")

but modify $pg_path to the real directory on your system. Don’t forget to start powershell with Administrator rights.

Set up apache maven

Metasfresh uses maven and even though Intellij Idea has maven built-in, we have noticed that if something breaks in intellij (eg. intellij shows an error that some file has compilation errors even though logically everything should be in order), it is much easier (and less time consuming) to recompile the metasfresh project from CLI using maven.

Download maven zip archive from https://maven.apache.org/download.cgi

Maven has no installer and there’s no need to add maven to PATH, so extract the archive to C:/ then rename the folder to C:/mvn from C:/apache-maven-3.6.3 (or whatever other version is currently available).

In the end you should end up with a folder structure on the disk such that mvn executable is found in C:\mvn\bin\mvn.cmd. Please note that C:\mvn\apache-maven-3.6.3\bin\mvn.cmd is the wrong path for this tutorial.

To test that maven is installed in the correct folder, run:

 C:\mvn\bin\mvn.cmd --version

and you should get output similar to:

PS C:\Users\crist> C:\mvn\bin\mvn.cmd --version
Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 2019-04-04T22:00:29+03:00)
Maven home: C:\mvn\bin\..
Java version: 1.8.0_212, vendor: Oracle Corporation, runtime: C:\Program Files\Java\jre1.8.0_212
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
PS C:\Users\crist>

Configure maven settings.xml file

Copy settings.xml from
C:\work-metas\metasfresh\misc\dev-support\maven\settings.xml
into
C:/Users/crist/.m2/settings.xml
file (create the .m2 folder if it doesn’t exist).

Compile the project using maven

This has to be done from the command line (powershell), because intellij idea’s Maven->Generate Sources and Update Folders for all Projects option doesn’t work for metasfresh. Some of the generated sources for external .xsd files are not created.
Here is the youtrack bug about this: https://youtrack.jetbrains.com/issue/IDEA-256661

This is a long-duration action, since we’re compiling the project and also downloading all the dependencies.

Run the following code in powershell (make sure to replace crist with your username:

$start = Date

$env:JAVA_HOME = "C:\Program Files\Java\jdk1.8.0_212"
$mvnSettingsFile = "C:\Users\crist\.m2\settings.xml"

cd C:\work-metas\metasfresh\misc\parent-pom\
C:\mvn\bin\mvn.cmd -T 2C -DskipTests --settings $mvnSettingsFile clean install

cd C:\work-metas\metasfresh\misc\de-metas-common\
C:\mvn\bin\mvn.cmd -T 2C -DskipTests --settings $mvnSettingsFile clean install

cd C:\work-metas\metasfresh\backend\
C:\mvn\bin\mvn.cmd -T 2C -DskipTests --settings $mvnSettingsFile clean install

echo "Elapsed: "

(Date) - $start

When this step is finished, you will see the following:

Elapsed:


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 7
Milliseconds      : 671
Ticks             : 76716038
TotalDays         : 8.87917106481481E-05
TotalHours        : 0.00213100105555556
TotalMinutes      : 0.127860063333333
TotalSeconds      : 7.6716038
TotalMilliseconds : 7671.6038

Install and set up docker

Installation

Follow the tutorial here to download and install docker: https://docs.docker.com/docker-for-windows/install/.

Note that you will have to restart your computer after the installation.

Install Jetbrains Toolbox and download Intellij Idea Ultimate

Download the installer from https://www.jetbrains.com/toolbox-app/.
We recommend using the Toolbox app as it helps tremendously with keeping intellij up to date, using different intellij idea versions for different projects and managing your projects.

Note: This guide is for Intellij IDEA Ultimate. Community edition might work.

We recommend the ultimate edition as it has additional features for web/spring applications.
E.g: autodetects spring and JEE applications and docker configurations, we have already setup data sources for various databases in metasfresh, and is also helpful for javascript development.

Configure maven in intellij

Start Intellij then go to Configure → Settings

In Build, Execution, Deployment -> Build Tools -> Maven make sure the path to the settings file is the one you downloaded previously.

In Build, Execution, Deployment -> Build Tools -> Maven -> Importing make sure the items are checked/unchecked as below.

If possible, select the JDK used for importing as Java 8 and not the internal JRE 11. If that is not possible, it will be done later.

Import metasfresh in intellij idea

Copy the intellij development settings

Copy the .idea folder from
C:\work-metas\metasfresh\misc\dev-support\intellij-idea\
into
C:\work-metas\.

In the end you should have a folder C:\work-metas\.idea which contains runConfigurations, codeStyles and other folders.

Copy the metasfresh development application.properties

Copy the metasfresh folder from

C:\work-metas\metasfresh\misc\dev-support\application.properties

into

C:\work-metas\

Merge the folders and override any existing files if asked.
This copies all the settings needed to connect to the docker instances of postgres, rabbitmq, and sets the correct java application ports.

Choose Projects -> Open

Select folder C:\work-metas

Now the importing starts. Intellij will show you multiple notifications. You can enable some of them:

Always download pre-build shared indexes:

Show Spring Boot run configurations in Services tool window:

You will see this toolwindow:

Open Project Structure via CtrlAltShiftS.

• Under SDKs, make sure your JDK 8 is visible in Intellij. If it is not in that list, press the + button and add it.
  
• Under Project, make sure the Project SDK is yours
  
  Press, Apply, OK to submit.

Open Settings via CtrlAltS.

In Build, Execution, Deployment -> Build Tools -> Maven -> Importing select he JDK for importer: Project JDK.

Add the pom files

In the Project toolwindow right click metasfresh/backend/pom.xml and choose Add as Maven Project

This step will possibly take a long time, as all the dependencies needed by metasfresh will be downloaded.

The import process will give an error, because we need to add more poms to intellij:

• metasfresh/misc/parent-pom/pom.xml
• metasfresh/misc/de-metas-common/pom.xml

At this point, all maven modules should be imported successfully.

If you’re asked to Download pre-built shared indexes, choose Always download

Customize Intellij

Configure Services toolwindow

Press “4 squares” and disable Configuration -> Type

Press “plus” and chose “Run configuration type” and chose the following:

• Docker
• npm
• Remote JVM Debug

In the end, you should see the following Run Configurations:

VCS on steroids

Press CtrlShiftA, type Registry and select the entry Registry...

Scroll down until the option: vcs.commit.tool.window appears and uncheck it!

Configure intellij to work with docker

Open Settings via CtrlAltS.

In Build, Execution, Deployment -> Docker press + and chose option Docker for Windows. You should see a Connection successful` message.

Press Apply.

Configure intellij to work with node

Open Settings via CtrlAltS.

In Language & Frameworks -> Node.js and NPM ensure that Node interpreter and Package manager are the ones you installed.

Press Apply Ok.

Run metasfresh from Intellij

Install infrastructure: postgres and rabbitMQ in docker

In Services toolwindow, select infrastructure and select Run:

At the end you should see the message: Compose: infrastructure' has been deployed successfully.

Install dependencies for frontend

In Project toolwindow, select the file metasfresh/frontend/package.json, right click and run npm install:

Copy the file
metasfresh\frontend\config.js.dist
into
metasfresh\frontend\config.js
without changing anything in it.

after_import.sql

Ask the metasfresh team for our after_import.sql secret sauce for developing on metasfresh and creating valid migration scripts. You need to run this mmigration script manually.

Run configuration Run SQL Migrations

This is needed after every git pull from master. It updates the local database with changes from your coleagues.

In Services toolwindow, select folder all local, right click and select Run. The 2 java classes, along with the frontend should start.

You’re finished!

We await your PRs. Good luck !

---

Optional (yet suggested) steps

Use Eclipser plugin

• There are some eclipse launcher files in the project. You could install eclipser plugin and
  convert the files whenever you need them. Just right click on a “.launch” file and you’ll find the Convert with eclipser option.

Eclipse and IntelliJ IDEA interop

Eclipse can only find // todo comments if todo is in caps rage. To do this you should go to Settings -> Editor -> Live Templates and add a new template as follows:

Learn your tools

It pains me greatly when i see people using the mouse and searching through menus to accomplish some action. Most often than not your IDE has a keyboard shortcut for doing a particular task. USE. IT.

• Read all the Tip of the Day tips
• Install the plugin IDE Features Trainer and do the tutorials there
• use the shortcut Ctrl+Shift+A
• use the shortcut Shift+Shift
• let the IDE do refactorings for you, instead of editing everything manually

[TheBestPessimist]

Just a week and will test this for myself

[metas-ts]

@Hanelore

Please go to https://projectlombok.org/setup/intellij and install lombok

I just saw a message where intelliJ recommended me to enable annotation processing and guided me to this setting:

@Hanelore I now activated it…did that make sense?
I’m not yet sure which parts of the compiling are really done by “native” maven (I guess there, the lombok related processing is on as per pom.xml) and where a setting like this one makes a difference (Note: I didn’t yet get to try and run anything from IntelliJ)

[lenscuba90]

Hi everyone, I am a newbie to metasfresh/java and spring boot framework, but I am most interested in this project and want to learn from it.

If someone could just clarify editing the settings.xml file to get the source code into Intelij ultimate I would be grateful.

Thus far using a Kubuntu 18.04 machine I have navigated using Dolphin from root/home/len/ than used control tab to unhide folders and files in my home dir. The path to the .xml file is Home/.m2/wrapper/dists/apache-maven-3.6.0-bin/2dakv70gp803gtm5ve1ufmvttn/apache-maven-3.6.0/conf/settings.xml

This is what the file looks like-

<!–
pluginGroup
| Specifies a further group identifier to use for plugin lookup.
com.your.plugins

–><!–
proxy
| Specification for one proxy, to be used in connecting to the network.
|

optional
true
http
proxyuser
proxypass
proxy.host.net
80
local.net|some.host.com

–><!–
server
| Specifies the authentication information to use when connecting to a particular server, identified by
| a unique name within the system (referred to by the ‘id’ attribute below).
|
| NOTE: You should either specify username/password OR privateKey/passphrase, since these pairings are
| used together.
|

deploymentRepo
repouser
repopwd

–><!–
Another sample, using keys to authenticate.

siteServer
/path/to/private/key
optional; leave empty if not used.

–><!–
mirror
| Specifies a repository mirror site to use instead of a given repository. The repository that
| this mirror serves has an ID that matches the mirrorOf element of this mirror. IDs are used
| for inheritance and direct lookup purposes, and must be unique across the set of mirrors.
|

mirrorId
repositoryId
Human Readable Name for this Mirror.
http://my.repository.com/repo/path

–><!–
profile
| Specifies a set of introductions to the build process, to be activated using one or more of the
| mechanisms described above. For inheritance purposes, and to activate profiles via
| or the command line, profiles have to have an ID that is unique.
|
| An encouraged best practice for profile identification is to use a consistent naming convention
| for profiles, such as ‘env-dev’, ‘env-test’, ‘env-production’, ‘user-jdcasey’, ‘user-brett’, etc.
| This will make it more intuitive to understand what the set of introduced profiles is attempting
| to accomplish, particularly when you only have a list of profile id’s for debug.
|
| This profile example uses the JDK version to trigger activation, and provides a JDK-specific repo.

jdk-1.4

  <activation>
    <jdk>1.4</jdk>
  </activation>

  <repositories>
    <repository>
      <id>jdk14</id>
      <name>Repository for JDK 1.4 builds</name>
      <url>http://www.myhost.com/maven/jdk14</url>
      <layout>default</layout>
      <snapshotPolicy>always</snapshotPolicy>
    </repository>
  </repositories>
</profile>

–><!–

 | Here is another profile, activated by the system property 'target-env' with a value of 'dev',
 | which provides a specific path to the Tomcat instance. To use this, your plugin configuration
 | might hypothetically look like:
 |
 | ...
 | <plugin>
 |   <groupId>org.myco.myplugins</groupId>
 |   <artifactId>myplugin</artifactId>
 |
 |   <configuration>
 |     <tomcatLocation>${tomcatPath}</tomcatLocation>
 |   </configuration>
 | </plugin>
 | ...
 |
 | NOTE: If you just wanted to inject this configuration whenever someone set 'target-env' to
 |       anything, you could just leave off the <value/> inside the activation-property.
 |
<profile>
  <id>env-dev</id>

  <activation>
    <property>
      <name>target-env</name>
      <value>dev</value>
    </property>
  </activation>

  <properties>
    <tomcatPath>/path/to/tomcat/instance</tomcatPath>
  </properties>
</profile>

–><!–
activeProfiles
| List of profiles that are active for all builds.
|

alwaysActiveProfile
anotherAlwaysActiveProfile

–>

Am I to copy/edit in the additional code provided for the Intelij IDE above the last of the code in this file --> above??? and thank you for your assistance in advance.

[nelucristian]

Hello,
I’m Cristian and I’m new to metasfresh community and i’m happy i joined it. I started to follow this setup and i’m blocked at step 5. I installed i think all the modules from metasfresh and also metasfresh-webui-api but InteliJ doesn’t detect Spring Boot run configurations. Can someone please help me with this setup?

Some specs about my PC:

• Windows 10 Pro
• InteliJ IDEA Community
• CPU i5
• RAM 8GB

[TheBestPessimist]

Hello @nelucristian,

Idea Community edition does not support Spring, only Ultimate version does (ref: JetBrains Products Comparison).

What you could do is:

• try to run the class WebRestApiApplication w/o spring config but with the needed arguments – maybe that works, i’m not sure as i have Idea Ultimate version
• buy and use Idea Ultimate version
• use eclipse, for which we have a guide at Developer's Tag Index

Here’s the relevant config i have in my idea:

[nelucristian]

Thank you very much. I made changes at the run configuration which you specified by screenshots and now WebRestApiApplication it starting also in InteliJ IDEA Community.

I can not start unfourtunatley ServerBoot because it fails at compilation stage. OutOfMemory error when compile ‘ModelClassGenerator’. I have just 8 GB of RAM in my PC. I will upgrade to 16GB

[adi-stefan]

When having trouble running the .launch files via Eclipser in Intellij (because of eclipse-specific path references), I was able to solve them by using the following absolute paths: