Use tomcat embedded with custom versions

Rationale TL;DR

I’ve been trying to shp a Java application as an executable WAR file (yes, I know about jar files, but I don’t want to skip one final monster). To do so I need to embed Tomcat on the WAR file.

Using Spring Boot starter web

Usually that would be as easy as using the spring-boot-starter-web artifact. It will pull the three tomcat-embed-core, tomcat-embed-el and tomcat-embed-websocket. So far so good. The issue here is that those will be in sync with the Spring Boot you’re using, so what if you’re still on 1.5.x but you want to use Tomcat 9.x? Or what if you’re already on SB 2.x but you want Tomcat 8.5.x?

Selecting the Tomcat version to run

You need to change your pom file to include the following:

<dependency>
  <groupId>org.apache.tomcat.embed</groupId>
  <artifactId>tomcat-embed-core</artifactId>
  <version>${tomcat.version}</version>
  <scope>compile</scope>
</dependency>
<dependency>
  <groupId>org.apache.tomcat.embed</groupId>
  <artifactId>tomcat-embed-el</artifactId>
  <version>${tomcat.version}</version>
  <scope>compile</scope>
</dependency>
<dependency>
  <groupId>org.apache.tomcat.embed</groupId>
  <artifactId>tomcat-embed-websocket</artifactId>
  <version>${tomcat.version}</version>
  <scope>compile</scope>
</dependency>

That would pull those dependencies from the maven central repository. Note that you need to define the property ${tomcat.version} somewhere in the properties block in your pom file.

Deal with geo_point types on ElasticSearch

Rationale TL;DR

I’ve been trying to ship some logs from Nginx to ERK (Elastic + Rsyslog + Kibana). The process is easy enough (I’ll post about that eventually). But, I ran into some problems when using the mmdblookup rsyslog module to do geolocalization because I wanted to position the remote clients on Kibana.

The solution

The issue is that out of the box ElasticSearch is unable to figure out that a JSON key like this:

"geo": {
	"loc": {
		"lat": 0.0,
		"lon": 0.0,
	}
}

The issue here is that ElasticSearch will simply figure out those fields are floats (lat and lon will be floats). If that’s the case, Kibana won’t be able to use them because it needs the loc field to be geo_point.

Every index uses a mapping that helps ES to figure out what to do with the data. Changing the mapping applied to loc would fix the issue.

However, using a mapping to set the loc field did not fix the problem, it make things even worse, because:

• We can’t just set the mapping of one field, if we do that ES will just use that mapping for that field and will not process the other. That is to say that we’re saying ES that we’re just worried about on field.
• Since ES 6.x we can’t delete a mapping without deleting the index itself. That means we need to reindex all the data, drop the index (and the mapping) and the reindex back to our original index.

So, we need something else. That’s what index templates are used for. They allow us to set the defaults applied to an index, including the mappings to apply. We can have multiple templates, deciding in which order they will be applied. The only drawback is that they need to applied at creation time.

Let’s suppose we want to apply a template to an index named nginx_logs-*. We want to set the field geo_ip.loc so that ES is able to use that field as a geo_point field:

PUT _template/nginx_geo_point
{
  "order": 0,
  "index_patterns": "nginx_logs-*",
  "settings": {
    "number_of_shards": 5
  },
  "mappings": {
    "_default_": {
      "properties": {
        "geo_ip.loc": {
          "type": "geo_point"
        }
      }
    }
  }
}

The syntax is clear. This creates a new template called nginx_geo_point. The index will use 5 shards and a new mapping will be applied, setting geo_ip.loc to geo_point.

Also, templates are applied in order. The order field controls that. In this example, this template will be applied the first one. If multiple templates have the same order number, there’s no guarantee of the order they will be applied.

Embed info in the Tomcat logs using Logback

Rationale

Let’s supose you are using CI/CD and you want to embed the build number in your logs. This is a good practice because, while checking the logs, that would allow you to find out what version was having that problems. However, we would like that to happen automagically from our CI/CD.

Let’s suppose we’re running Jenkins. If you’re using other CI/CD solutions like TravisCI or CircleCI your milleage will certainly vary.

The solution

To make this easier, we would like to start from maven. The idea would be to call maven with an additional variable -Dbuild.number=${BUILD_NUMBER}. What happens if that variable is not being provided? Then maven will automatically use a default value (maybe 1?).

After that, maven should ‘save’ that value somewhere for logback to pick up. How do we do that? We can ask maven to patch a placeholder in the application.properties file.

Finally, because we’re using Spring Boot you would need to pull in spring-boot-starter-logging. It is included by default in spring-boot-starter so probably you are good to go! We can then use the ${build.number} variable inside the logback xml configuration and log the build number.

Using the BUILD_NUMBER Jenkins variable

The ${BUILD_NUMBER} variable is already available when using Jenkins. Nothing to do here but call maven passing that value:

mvn -Dbuild.number=${BUILD_NUMBER} 

Inside the parent pom file, under the properties tag we need to add something like this:

<build.number>1</build.number>

Patching the application properties Spring Boot file

You need to use the filtering maven plugin and instruct it to patch files in a given directory. For example, in my case my various (depending on the environment) application-$env.yml files are under src/main/resources:

   <resource>
      <directory>src/main/resources</directory>
      <filtering>true</filtering>
   </resource>

The relevant stuff you’d need to add to your application.yml file to have the build number there are:

build.number: @build.number@

Maven will replace the @build.number with whatever value you passed to maven (or the default value).

Picking up the build number with Logback

Finally we need to pick up the build number value in the application.yml file. Because Logback loads the logback.xml before Spring Boot’s Application context is there, you need to rename your logback.xml to logback-spring.xml. The later one is being processed after Spring Boot magic happens, so then the application.yml is being picked and the build.number is available to Logback to use. According to the documentation [1]:

When possible, we recommend that you use the -spring variants for your logging configuration (for example, logback-spring.xml rather than logback.xml). If you use standard configuration locations, Spring cannot completely control log initialization.

Remember, you need to use spring-boot-starter-logging, if you are using spring-boot-starter then you already have it in place.

The only thing you need is to use the build number variable. In my case:

<Pattern>%date{yyyy-MM-dd'T'HH:mm:ss.SSSZ} [%thread] [build.number:${build.number}] [%X{user}] %-5level %marker       %logger{36} - %msg %throwable{full}%n</Pattern>

This makes it really easy to parse the message and filter by build number.

[1] https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-logging.html