Functionality Decoupling into a Microservice by a simple example

23 March 2023

This post will be more of my hypothetical deliberation on splitting a monolith. I stumbled multiple times upon a software that was logically split in code into separate modules and programs but still used the same database, often read and write from common tables. Today, using a simple example we will see how it can be separated.

First draft of the example application

Let us consider a sample application. It has a mechanism for authentication, authorization of users as well as some fields that can be updated by the user. The user will be described by the following schema:

type User struct {
    ID        int64     // Typical ID
    Name      string    // Some username, can be e-mail
    Password  string    // Password hash
    Super     bool      // Account status - premium/gold however you call it
}

What is more, the user can choose their favorite fruit in another table but the list is predefined and Super user is the only one that can select pineapple.

type Fruit struct {
    ID        int64     // Foreign key ID
    Name      string    // Name of the fruit
}

The constraints can be hardcoded into the application using if statements or a list kept in some config file. In the example below, the functions IsFruit, IsFruitSpecial and InsertOrUpdateFruit are implemented "somewhere".

// on: PUT into /fruits
func (u *User) UpdateFruit(fruit String) error {
    if ! IsFruit(fruit) {
        return fmt.Errorf("Fruit %s is not in the list of available fruits!", fruit)
    }

    if u.Super && IsFruitSpecial(fruit) {
        return fmt.Errorf("Only Super users can select special fruits like %s!", fruit)
    }

    return InsertOrUpdateFruit(u.ID, fruit)
}

To put it into rough graphics, that is how our application looks like currently:

First version of the monolith service

Follow to this repository to access the first version of our service. (It's just an example, there is a lot to improve there!) Use Go to build it with go build and run ./monolith to use it. Create some users with:

$ ./monolith -username user1 -password password123

The first user you create will have ID = 1 and have no super status. Each next user will have ID incremented by 1 and will also have no super status. By default there are no fruits set.

Default SQLite database name is monolith.db in the current working directory.

You can switch each user's status using these commands (where 1 is the user ID):

$ ./monolith -super 1
$ ./monolith -unsuper 1

To start the server on port :8080, use ./monolith -serve. Then you can access the website in your browser using http://localhost:8080. Users should be able to log in and set their favorite fruit from the combo box. Fruits marked with asterisk (*) are special and can only be used by super users.

Code of this service is categorized between different directories:

• users handles user creation, setting status, checking status, session creation, verification and deletion,
• fruits handles fruit setting, listing the fruits and checking if the fruit is special. It is currently tightly coupled with the users package, due to JOIN statement when showing the fruits (mapping user ID to username),
• router handles the routing HTTP endpoints, session activation based on cookie, redirects, etc. It uses function from both users and fruits.
• main file is fortunately not referencing anything from the fruits package. It only calls the router to serve the website, create users and set their status.

Our goal is to decouple the fruits from the entire codebase and make it a new microservice that can run independently.

Target structure

So, our plan is to split up the application into the old monolith and a new microservice that will handle favorite fruits. The new microservice will have two functions - getting the list of all fruits of all users, changing user's favorite fruit based on authorization token, and we will extend it with function of getting fruit of a single user.

What is more the new microservice will implement the functions for checking if the input is an actual fruit and if the selected fruit can be set by the user.

The old monolith service will be still responsible for user and session management but also for generating a token (see next paragraph).

How will the fruits service know if the user is authenticated or super?

Here comes the magic of cryptography. For this matter we will use JSON Web Token that is a base64 encoded JSON object along with a cryptographic signature. Only the old accounts service will be able to sign the token with its private key. Then fruits service will use the old service's public key to verify the token's validity. Even though we use cryptography here, data in the tokens is not encrypted (potentially only in transit with TLS). Stealing the token from the user is as risky as stealing session cookie. However, signature protects the token from being manipulated, so we will include an expiration time in it.

// Inside accounts/old service
func CreateToken(id int, super bool) (string, error) {
    // Create a custom token with our needed fields
    claims := struct {
        UserId int
        Super  bool
        jwt.StandardClaims
    }{
        id,
        super,
        jwt.StandardClaims{
            ExpiresAt: time.Now().Unix() + 10, // 10 seconds
        },
    }

    // Generate the token structure and sign it with Ed25519
    token := jwt.NewWithClaims(jwt.SigningMethodEdDSA, claims)
    return token.SignedString(private_key)
}

// Inside fruits/new service
func VerifyToken(token string) (int, bool, error) {
    claims := struct {
        UserId int
        Super  bool
        jwt.StandardClaims
    }{}

    // Parse the JSON object into the struct and verify the signature with public key
    _, err := jwt.ParseWithClaims(token, &claims, func(token *jwt.Token) (interface{}, error) {
        if _, ok := token.Method.(*jwt.SigningMethodEdDSA); !ok {
            return nil, fmt.Errorf("Not today, Mr Robot, no use for: %v", token.Header["alg"])
        }
        return public_key, nil
    })

    if err != nil {
        return 0, false, err
    }

    return claims.UserId, claims.Super, nil
}

Possible problems with this approach

The first problem that is obvious is that querying fruits it tightly coupled with the users table because of the JOIN statement. There are multiple possibilities to solve this problem:

1. Denormalization ¹⁴⁵ - store username along ID and selected fruit. This is by far the simplest approach. However, what if user changes the username? We would need to also update it in the fruits table. We can create an endpoint in the new microservice and instruct the old monolith part to call it when the username changes. But this solution breaks the idea of being loosely coupled and independent operation of microservices when one of the services is down. The only way here would be to emit a message to a message broker and hope that it will be picked up by fruits service.
2. Query the old service from the fruits service or query the new fruits microservice from the old monolith. Again, this goes against independence of the services when one of them is down. We don't want to show 503 errors too often. Also the amount of requests can become unmanageable if the lists are large. Chatty communication can lead to tight coupling. ²
3. Create a third service that will be responsible for joining together the list of fruits and users. This seems like a good solution. That way business logic will be kept within boundaries of users and fruits microservices, while router will be transformed into a proxy with some presentation logic. But in case the list of users and/or fruits is large, that it doesn't fit into memory, and we would need multiple calls to fetch the data, this issue will become complex and require caching implementation to keep the performance to the maximum. ³⁶

Another problem with this design is when we want to create a bugfix that removes the fruit setting from the table based on up-to-date user's status. For example, what if the user suddenly becomes not super? In a monolith, setting the logic might would be simpler - just query fruits table and verify if the selection is still valid. For the microservice, we will need to somehow notify the fruits service about the change. This might be immediate or scheduled, for example once per day make synchronization between two services. However, this causes us to need to implement another interface. We can also use a message broker like RabbitMQ, broadcast the change from the accounts service and let fruits service poll the topic, just like in the first solution.

In the next post, we will go through the process of decoupling the service using the first approach. We will store displayName along with the user's id and fruit choice.