When end-2-end testing applications that use an IdP, an IdP needs to be in a known state to make test repeatable.

Typically a container is used, with a configuration that needs to be reset before (and after) a run. Restoring the IdP configuration isn't ideal, since addring new test cases (e.g. adding a user with different properties to check application behavior). I propose a different approach: One-off IdP

Container without persistence

I start with an empty deployment of KeyCloak running in a docker container.

#!/bin/bash
#Run a clean KeyCloak
docker run --rm -p 8080:8080 \
       --name testcloak \
       -e KEYCLOAK_ADMIN=admin \
       -e KEYCLOAK_ADMIN_PASSWORD=password \
       quay.io/keycloak/keycloak:latest start-dev

The --rm parameter ensures that the container is discarded after use. There is no persistence flag (--mount), so when the container goes down, all data perishes (and that's intendet).

Configuration sequence

The empty KeyCloak only knows the realm master and the user admin. To turn it into a fully functional IdP we need to configure it. Since we want this process to be repeatable we shall use Keycloak's REST API. The documentation is complete, including an OpenAPI spec, but in a dictionary style, so all is good when you know what you are looking for. To learn what is needed the browser development tools while using the admin UI teach us the what.

The Domino REST API delivers collections using chunked transfer encoding. This has the advantage, that you can process results as they arrive. It produces the challenge that the usual client side code is designed to first wait for completion of the request. I wrote about the JavaScript solution a while ago, this is the Java edition.

Client choices

In JavaScript land the choice of client is simple: the Fetch API. In Java we have some choices:

• Since Java 11 there's a HttpClient in the JDK
• A very popular library is provided by the Apache HTTP components
• The SPRING framework, undisputed king of Java web frameworks, offers its own sset of clients
• The vert.x framework, which we used to build DRAPI, features its own client

There are probably more around. This article uses the JDK HttpClient. I'll skip the parts with Authentication and TLS handling, check the full example for details.

How it works

First we create an java.net.http.HttpClient. It takes care of the http version and the TLS context.

HttpClient getClient(SSLContext sslContext) {
  return HttpClient.newBuilder()
           .sslContext(sslContext)
           .build();
}

Then we build and execute the request. The magic is the BodySubscriber (more on that below).

Integer runGetRequest(HttpClient client, String url, String authHeader, BodySubscriber subscriber) throws Exception {
  HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .header("Authorization", authHeader)
            .GET()
            .build();

  CompletableFuture<Integer> response =
          client.sendAsync(request, responseInfo -> subscriber)
          .whenComplete((r, t) -> System.out.println("Response: " + r.statusCode()))
          .thenApply(HttpResponse::body);

  return response.get();
}

You are developing a web application using Quarkus that consists of multiple (micro)services and a bunch of supporting libraries. Since maven modules provide sufficient isolation, you decide to use a parent project to keep dependency versions and parameters in sync and a Maven Reactor to build them together.

This blog post is for you. A special thanks to Alexey for helping out.

Moving parts

Our objective is to have a development setup where we can edit any of the services or libraries and then run them individually or all together. Ideally without the need to alter configurations between runs and the ability to deploy the setup using devcontainers (note: that's about the development setup, not about deploying the finished application). There are some moving parts:

• Quarkus CLI or Maven plugin
• Multi-Module Maven to keep libraries and services together and apart
• Maven profiles to conditionally modify maven runs
• Maven's <pluginManagement> and <dependencyManagement> to define configurations without activating them
• Your favourite Ide configured for Java and Quarkus
• Understand how VSCode's Tasks work
• Know how to setup DevContainers, especially when you need sidecars

That's a lot, let's dig in.

In the beginning was a test, that provided to be vital, but not sufficient. In modern application development we encounter:

• Unit Tests: testing a single function or a tuple of related functions
• Module Tests: testing bigger parts of your application, without actual external dependencies
• Integration or End-to-End tests: a.k.a life firing exercise

This entry isn't a discussion about the merits of how much and when test, but making tests easy to setup and distinguish

The manual way

We typically use Mockito, vert.x and REST-assured in our tests, so a typical test class would look like this:

@ExtendWith(VertxExtension.class, MockitoExtension.class, MyCustomExtension.class})
@Tag("UnitTest")
class SomethingTest {

  @Test
  does_it_blend() {
    // Test goes here
  }
}

It is just two lines, but everywhere. You can simplify it by creating your own annotation.

The custom annotation

@Target({TYPE, METHOD, ANNOTATION_TYPE})
@Retention(RUNTIME)
@ExtendWith({VertxExtension.class, MockitoExtension.class, MyCustomExtension.class})
@Tag("UnitTest")
public @interface UnitTest {
  // no action needed here, JUnit use only!
}

Now you simply use:

@UnitTest
class SomethingTest {

  @Test
  does_it_blend() {
    // Test goes here
  }
}

While this looks like minor cosmetic, it allows to control test extensions from a single place, your annotation source. Repeat that process for the other test types (ModuleTests, IntegrationTests, PerformanceTests etc.) you want to use.

In your pom.xml, in the build-plugins section you can use the tag to ensure all your unit test, but only them execute on mvn test and the others on mvn verify

<plugin>
  <!-- Run UNIT and MODULE tests, no backend calls -->
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <groups>UnitTest,ModuleTest</groups>
  </configuration>
</plugin>

As usual YMMV

Know your patterns, so you don't re-invent the wheel. Patterns are like LEGO: You can combine them to larger patterns. One of those is the Hexagonal Achitecture officially called:

Ports and Adapters

proposed by Alistair Cockburn, one of the co-signers of the Agile Manifesto.

I've written about his work before and can highly recommend his latest book "Hexagonal Architecture Explained" (or for that matter any of his writings).

In a nushell, Ports and Adapters proposes an approach that leads to loosely coupled systems, that separates business logic from the software environment (think I/O, databses, API and UI) to facilitate clarity and ease of testing.

This post doesn't aim to explain it, for that you can refer to the Wikipedia article or, better, read the book but trying to describe how to apply it to Java in a Maven project.

Accessing applications usually entails some kind of identity. Some part(s) of your application provide identity (called IdP), while other's consume it (paraphrased from Captain Obvious). Identity could be provided from a record or document in your or another database, an LDAP directory, an OICD or a 3d party like your eMail provider or social account, or with some hoops and loops Webauthn (a.k.a passkey).

The question is: how deep does it go?

A typical archtecture

For simplicity I'm skipping load balancers and cache facilities

Each tier might or might not have its own identity, lets have a closer look

(0) User tier

When is authentication information stored at the user tier (think Browser, app or OS) and how immediate is it? Like a username/password or a passkey private key. Those can be exchanged for a token. Or is it a session cookie or an access_token allowing immediate access? How are those credentials protected and/or synced? How vulnerable are they on physical access?

(1) Access tier

A.k.a the firewall or VPN Does it assert a valid user? Are the credentials the same or different from your application. Does your access tier include IP ranges or geo location as part of identity? I've seen identity requirements 100% on VPN, but rarely on firewalls

(2) Web tier

TYpically you find a static web server like nginx, Apache http, the Kubernetes Ingress or a service by a cloud provider.

Even when all your static resources are served by your application tier, you can identify your web tier where requests flow through. When you can establish identity there (and reject invalid ones), you have one more protective layer. nginx+ can do that with JWT

(3) Application tier

YOur application could be a monolith, microlith, microservice, follow a layered architecture, be message driven, event driven or be contemporary with the hexagonal architecture. In any case your user facing access will require identity.

It becomes blurry when your user facing services then call out to other services (via http or message/event), what identity are they using to communicate: user, service or both? You might start looking at RBAC. In any case, this needs to be planned carefully

(4) Persistence / database tier

The prevalent examples you find online , especially in the realm of DBaaS, use just one service identity to access your persistence (file system, database , imp, etc.). So database logs won't tell you who accessed data (RW/RO) and you won't have a chance to implement row level security.

Interestingly this isn't a limitation of databases, they all come with user management, but rather the headache maintaining it or setup another auth

When we designed the Domino REST API, we decided to stick to the JWT based user identity all the way to the database. Apache CouchDB also allows for JWT based authentication. It was lacking the ability to point to an IdP's jwks, so I contributed the CouchDB IdP updater, go check it out.

How do you use identity?

A question that pops up frequently in developer discussions is "how to structure your data in a NoSQL way?". To shed a light on this, we have a look at the approach invented 50 years ago and still an all time favorite

Normalization

In a simple order example, we are looking at four tables:

• Customer
• Product
• Order
• OrderEntry

In this design, there are no duplicates and some simple SQL can list out all I need, for example the order value for a given order:

SELECT oi.order_id,
       SUM(oi.quantity * p.price) AS order_total
FROM OrderItem oi
JOIN Product p ON oi.product_id = p.id
GROUP BY oi.order_id
WHERE oi.order_id = 67111;

or the revenue per customer:

SELECT c.id AS customer_id,
       c.name AS customer_name,
       SUM(oi.quantity * p.price) AS revenue
FROM Customer c
JOIN Order o ON c.id = o.customer_id
JOIN OrderItem oi ON o.id = oi.order_id
JOIN Product p ON oi.product_id = p.id
GROUP BY c.id, c.name;

Development Containers are supposed to liberate your development environment from a specific local installation, like container technology liberated your runtimes (a.k.a YAMLed them into Docker or Kubernetes).

Development != Runtime

Containerization for development has some overlap and quite some difference to containerization for production:

Insights

• There are many getting started resources available: here, here, here, here and here. They are working examples, commonly strong on what and how, but light on why
• There are plenty of templates to learn by example
• There seem to be substantial differences how it works on different platforms, subtle and annoying
• On macOS (14.4.1) with with the devcontainer plugin 0.364.0 mount binds would not work in auxiliary containers, only in the main app
• I couldn't find any descrption which subset of docker-compose.yml is supported
• The most interesting templates, for now, live on the Microsoft Artifact Registry, when you know how to look. Starting with those saves you tons of time and frustration
• You will love code --list-extensions to figure out the extensions you have in your vscode (unless you are a code n00b and don't have any)

Imagine, you are tasked with maintaining a Java application that needs to run on more than one Java version. You want to ensure that it compiles, tests and builds on all of them.
This is our story, buckle up, there are a few moving parts

The big picture

• We use Apache Maven to drive the project using the pom.xml
• The Maven Toolchains plugin controls the Java versions
• Using <properties> ... </properties> and Build Profiles to adjust conditions for processing
• Annotatiosn like @Only8 and @Only17 help to qualify tests
• Our build tool (Jenkins or Github Actions) will use a container provided (in our case based on Redhat UBI)

In June I wrote about how to use Docker & nginx to deliver statically rendered brotli files for your web (frontend) application. It improves delivery quite a bid, but left me wonder: isn't there too much static WebServer involved?

Double hop to deliver static files

A typical web application using micro/mini/midi services looks like this:

It is common, easy and concerns quite separated. However it comes with a set of challenges:

• nginx doesn't do http/2 on proxy_pass, so you miss the ability to serve static files directly with http/2
• For static file we have two nginx involved
• Each service needs to be exposed to the host at some port
• The service architecture leaks to the host based nginx. SO any change in service needs an update to the docker-compose.yml AND the host based nginx configuration
• the containers depend on that, external to them, configuration

So I tried to design a better way to handle this without going all K-in:

This looed like a more promising approach:

• Services could be addressed with their internal network name
• Only Ports 80 and 443 of one container need exposure on the host
• The nginx configuration inside the container is immutable and can't accidentially be reconfigured in production (your image comes from a pipeline isn't it)

Challenges

• When trying to configure certbot, I initially tried using the --nginx parameter with an http challenge and shared data mounts. None of the attempts worked satisfactory, so at the end I settled on aDNS-01 using CloudFlare.
• Since I wanted the nginx configuration to be inside the container image (and not on a data mount), a good understandig of nginx's configuration is necessary. The only persisted information was /etc/letsencrypt for the certificate and a secret for CloudFlare credentials
• When the nginx configuration is statically configured for TLS, on initial load it will fail since the certs don't exist yet. Auntie Google suggested a manual run of certbot, but I favour docker compose up to handle everything
• I ended up creating my own docker images, which was an epiphany: it absolutely makes sense to build a container image for single use instead of trying hard to make it configurable and vulnerable to mis-configuration