Creating a microservice with REST API¤

Warning

Under construction!

In the previous tutorial, you learned how to create a web server.

In this tutorial, you'll learn how to create a basic ASAB microservice that provides a REST HTTP API. This microservice will implement CREATE, READ, UPDATE and DELETE functionality, in another words: CRUD.

We will also use MongoDB as a database running on Docker. If you're not familiar with CRUD, MongoDB, or Docker, take the time to learn the basics, because they are the foundations of backend programming with ASAB.

Set up the project with Docker and MongoDB¤

Install Docker if you don't have it already. Go to the official website, choose your operating system, and follow the guide.

Tip

Based on our experience, we recommend installing just the Docker engine - there's no need to install Docker Desktop in most cases. In some cases, installing Docker Desktop might even cause problems when interacting with Docker in the terminal.

You can always check if you have Docker installed successfully:
```
docker --version
```

Pull the MongoDB image from Docker Hub.

docker pull mongo

Start the container:

docker run -d -p 27017:27017 \
    -e MONGO_INITDB_ROOT_USERNAME=user \
    -e MONGO_INITDB_ROOT_PASSWORD=secret \
    mongo

Install Postman in case you do not have it (download here on the official website). We will use Postman to test the webservice REST API. In Postman, you can create your collection of HTTP requests, save the HTTPs requests as templates, or automatically generate documentation.

Prepare the structure of the project. Every ASAB microservice consists of several Python modules. Create the following file structure in your repository:

.
└── my_rest_api.py
─── my_rest_api
    └── __init__.py
    ─── app.py
    ─── tutorial
        └── handler.py
        └── service.py
─── conf
    └── config.ini

Build a microservice¤

Now that you've prepared the modules, we move on to actual coding. Here is the code for every module with explanations.

my_rest_api.py

#!/usr/bin/env python3  # (1)!
from my_rest_api import TutorialApp  The asab application will be stored in my_rest_api module.


if __name__ == '__main__':  As always, we start the application by creating it's singleton and executing the  run() method.

    app = TutorialApp()   
    app.run()

my_rest_api/app.py

import asab 
As always, let's start with importing the asab modules.

import asab.web
import asab.web.rest
import asab.storage 
This module is a built-in ASAB service that's used for manipulating items in databases.


class TutorialApp(asab.Application):

    def __init__(self):
        super().__init__() 
Remember, if you override a __init__() method, don't forget to add this super-initialization line of code.


        # Register modules
        self.add_module(asab.web.Module) 
asab modules are registered with the add_module() method on the asab.Application object.

        self.add_module(asab.storage.Module)

        # Locate the web service  
        self.WebService = self.get_service("asab.WebService") 
To access the web service, use the get_service() method. It is common to have all the services stored as attributes on the asab.Application object.

        self.WebContainer = asab.web.WebContainer(
            self.WebService, "web" 
TODO

        )
        self.WebContainer.WebApp.middlewares.append(
            asab.web.rest.JsonExceptionMiddleware 
TODO

        )

        
ASAB microservices consist of two parts: services and handlers. 
When HTTP request is sent to the web server, the handler identifies the HTTP request type and calls the corresponding service. The service performs its specified operations and returns data back to the handler, which sends it back to the client.

        from .tutorial.handler import CRUDWebHandler
        from .tutorial.service import CRUDService
        self.CRUDService = CRUDService(self)
        self.CRUDWebHandler = CRUDWebHandler(
            self, self.CRUDService
        )

Continue with the init file, so that the directory my_rest_api will work as a module.

my_rest_api/__init__.py

from .app import TutorialApp

__all__ = [
    "TutorialApp", The list of strings that define what variables have to be imported to another file. If you don't know what that means, this explanation might help. In this case, we only want to import TutorialApp class.

]

Create a handler¤

The handler is where HTTP REST calls are handled and transformed into the actual (internal) service calls. From another perspective, the handler should contain only translation between REST calls and the service interface. No actual 'business logic' should be here. We strongly recommend building these CRUD methods one by one and testing each one immediately.

my_rest_api/tutorial/handler.py

import asab
import asab.web.rest


class CRUDWebHandler(object):
    def __init__(self, app, mongo_svc):
        self.CRUDService = mongo_svc 
This is a reference to the microservice itself. In this app, we will create a service which uses the MongoDB Storage.

        web_app = app.WebContainer.WebApp

        # Handle PUT and GET requests
        web_app.router.add_put(
            '/crud-myrestapi/{collection}', 
Methods add_put, add_get take arguments path, which specifies the endpoint, and handler, which is a coroutine that is called after the request is received from the client. In fact, these methods are performed on aiohttp web handler and are special cases of the add_route method. In this case, the the path is /crud-myrestapi/{collection}, where collection is a variable name.

            self.create
        )
        web_app.router.add_get(
            '/crud-myrestapi/{collection}/{id}',
            self.read
        )

        self.JSONSchema = { 
In order to prevent storing arbitrary data, we define a JSON schema. Now if the request data does not satisfy the format, the data cannot be posted to the database.

        'type': 'object',
            'properties': {
                '_id': {'type': 'string'},
                'name': {'type': 'string'},
                'age': {'type': 'number'},
                'job': {'type': 'string'}
            }
        }


    @asab.web.rest.json_schema_handler(self.JSONSchema) 
The JSON schema handler is used as a decorator and validates JSON documents by JSON schema. It takes either a dictionary with the schema itself (as in this example), or a string with the path for the JSON file to look at.

    async def create(self, request, *, json_data):
        collection = request.match_info['collection'] 
This method is used for matching data from the URI. They must be listed in the brackets such as {collection} on line 12.


        result = await self.CRUDService.create(
            collection, json_data 
After the request is sent and the handler calls the create method, it calls a method with the same name on the service, expecting from it to perform logic operations ("save json_data into database") and then return data back.

        )
        if result:  
            return asab.web.rest.json_response(
                request, {"result": "OK"} 
Now if the service has returned some data back, the handler will send a positive response to the client...

            )
        else:
            asab.web.rest.json_response(
                request, {"result": "FAILED"} 
...or negative if the service didn't return anything.

            )

    async def read(self, request):
        collection = request.match_info['collection'] 
Once again, obtain the data from the path.

        key = request.match_info['id']

        response = await self.CRUDService.read(
            collection, key 
After the GET request is sent, the handler calls the service to perform a method read(), expecting some data back.

        )
        return asab.web.rest.json_response(
            request, response 
Simply respond with the data found in the collection. If the data does not exist, the response will be empty.

        )

Let's start with two methods - PUT and GET - which allow us to write into database and check the record.

The handler only accepts the incoming requests and returns appropriate responses. All of the "logic", be it the specifics of the database connection, additional validations, or other operations, take place in the CRUDService.

Create a service¤

my_rest_api/tutorial/service.py

import asab
import asab.storage.exceptions
import logging

L = logging.getLogger(__name__)


class CRUDService(asab.Service):

    def __init__(self, app, service_name='crud.CRUDService'):
        super().__init__(app, service_name)
        self.MongoDBStorageService = app.get_service(
            "asab.StorageService"
        )

    async def create(self, collection, json_data):
        obj_id = json_data.pop("_id")

        cre = self.MongoDBStorageService.upsertor(
            collection, obj_id
        )

        for key, value in zip(
            json_data.keys(), json_data.values()
        ):
            cre.set(key, value)

        try:
            await cre.execute()
            return "OK"
        except asab.storage.exceptions.DuplicateError:
            L.warning(
                "Document you are trying to create already exists."
            )
            return None

    async def read(self, collection, obj_id):
        response = await self.MongoDBStorageService.get(
            collection, obj_id
        )
        return response

As mentioned above, this is where the inner workings of the microservice request processing are. Let's start, as usual, by importing the desired modules:

Now define the CRUDService class which inherits from the [asab.Service]{.title-ref} class.

[asab.Service]{.title-ref} is a lightweight yet powerful abstract class providing your object with 3 functionalities:

Name of the [asab.Service]{.title-ref} is registered in the app and can be called from the [app]{.title-ref} object anywhere in your code.
[asab.Service]{.title-ref} class implements [initialize()]{.title-ref} and [finalize()]{.title-ref} coroutines which help you to handle asynchronous operations in init and exit time of your application.
[asab.Service]{.title-ref} registers application object as [self.App]{.title-ref} for you.

[asab.StorageService]{.title-ref} initialized in [app.py]{.title-ref} as part of the [asab.storage.Module]{.title-ref} enables connection to MongoDB. Further on, two methods provide the handler with the desired functionalities.

Create a configuration file¤

conf/config.ini

[asab:storage]
type=mongodb
mongodb_uri='mongodb://mongouser:mongopassword@mongoipaddress:27017'
mongodb_database=mongodatabase

Testing the app¤

Now everything is prepared and we can test our application using Postman. Let's create a new collection named celebrities provided with some information.

Start the application
```
python my_rest_api.py -c conf/config.ini
```
The application is implicitly running on an http://localhost:8080/ port.
Open Postman and set a new request. First, try to add some data using the PUT method to the localhost:8080/crud-myrestapi/celebrities endpoint. Insert this JSON document into the request body:
```
{
"_id": "1",
"name": "Johnny Depp",
"age": 60,
"job": "actor"
}
```
Hopefully you received a status 200! Let's add one more.
```
{
"_id": "2",
"name": "Lady Gaga",
"age": 37,
"job": "singer"
}
```
Now let's test if we can request for some data. Use the GET method to the localhost:8080/crud-myrestapi/celebrities/1 endpoint, this time with no request body.

Now, what is the response?

Success

Up and running! Congratulation on your first ASAB microservice!
Failure

If you see the following message:
```
ModuleNotFoundError: No module named 'pymongo.mongo_replica_set_client'
```
This message means that there's a missing module, probably the motor library, which provides an asynchronous driver for MongoDB. Try to fix it:
```
pip install motor
```
Failure

If you see the following message:
```
OSError: [Errno 98] error while attempting to bind on address ('0.0.0.0', 8080): address already in use
```
This message means that the port is already used by some other application (have you exited the application from the previous tutorial?) To check what's running on your port, try:
```
lsof | grep LISTEN
```
or
```
lsof | grep localhost:8000
```
If you something similar to the following output:
```
python3   103203    user    7u     IPv4  1059624 0t0 TCP *:bbs (LISTEN)
```
This output means there's a running process using the port with ID 103203.

The first option is simply to stop the process:
```
kill -9 103203
```
(Replace the ID with the corresponding ID from the previous output.)

The second option is to add these lines into the configuration file:
conf/config.ini
```
[web]
listen=0.0.0.0:8081
```
If you run the app again, it should be running on http://localhost:8081/ port.
Question

What if you see no errors, but also no response at all?

Try to check the Mongo database credentials. Do your credentials in the config file fit the ones you entered when running the Mongo Docker image?

Summary¤

In this tutorial, you learned how to create a simple microservice provided with REST API.

Exercise 0: Store JSON schema in the file¤

In order to get used to the asab.web.rest.json_schema_handler() decorator, store the JSON schema in a separate file. Then, pass its path as an argument to the decorator.

Exercise 1: Implement `POST` and `DELETE` methods¤

For updating and deleting data from the database, implement the methods POST and DELETE.

Implement update() and delete() methods to the CRUDService class. Use

handler.py

[./myrestapi/tutorial/handler.py]{.title-ref}

import asab
import asab.web.rest


class CRUDWebHandler(object):
    def __init__(self, app, mongo_svc):
        self.CRUDService = mongo_svc
        web_app = app.WebContainer.WebApp
        web_app.router.add_put(
            '/crud-myrestapi/{collection}',
            self.create
        )
        web_app.router.add_get(
            '/crud-myrestapi/{collection}/{id}',
            self.read
        )
        web_app.router.add_put(
            '/crud-myrestapi/{collection}/{id}',
            self.update
        )
        web_app.router.add_delete(
            '/crud-myrestapi/{collection}/{id}',
            self.delete
        )


    @asab.web.rest.json_schema_handler({
        'type': 'object',
        'properties': {
            '_id': {'type': 'string'},
            'field1': {'type': 'string'},
            'field2': {'type': 'number'},
            'field3': {'type': 'number'}
        }})
    async def create(self, request, *, json_data):
        collection = request.match_info['collection']

        result = await self.CRUDService.create(
            collection, json_data
        )
        if result:
            return asab.web.rest.json_response(
                request, {"result": "OK"}
            )
        else:
            asab.web.rest.json_response(
                request, {"result": "FAILED"}
            )


    async def read(self, request):
        collection = request.match_info['collection']
        key = request.match_info['id']
        response = await self.CRUDService.read(
            collection, key
        )
        return asab.web.rest.json_response(
            request, response
        )


    @asab.web.rest.json_schema_handler({
        'type': 'object',
        'properties': {
            '_id': {'type': 'string'},
            'field1': {'type': 'string'},
            'field2': {'type': 'number'},
            'field3': {'type': 'number'}
        }})
    async def update(self, request, *, json_data):
        collection = request.match_info['collection']
        obj_id = request.match_info["id"]

        result = await self.CRUDService.update(
            collection, obj_id, json_data
        )
        if result:
            return asab.web.rest.json_response(
                request, {"result": "OK"}
            )
        else:
            asab.web.rest.json_response(
                request, {"result": "FAILED"}
            )


    async def delete(self, request):
        collection = request.match_info['collection']
        obj_id = request.match_info["id"]
        result = await self.CRUDService.delete(
            collection, obj_id
        )

        if result:
            return asab.web.rest.json_response(
                request, {"result": "OK"}
            )
        else:
            asab.web.rest.json_response(
                request, {"result": "FAILED"}
            )

service.py

[./myrestapi/tutorial/service.py]{.title-ref}

import asab
import asab.storage.exceptions

import logging
#

L = logging.getLogger(__name__)

#


class CRUDService(asab.Service):

    def __init__(self, app, service_name='crud.CRUDService'):
        super().__init__(app, service_name)
        self.MongoDBStorageService = app.get_service(
            "asab.StorageService"
        )

    async def create(self, collection, json_data):
        obj_id = json_data.pop("_id")

        cre = self.MongoDBStorageService.upsertor(
            collection, obj_id
        )

        for key, value in zip(
            json_data.keys(), json_data.values()
        ):
            cre.set(key, value)

        try:
            await cre.execute()
            return "OK"
        except asab.storage.exceptions.DuplicateError:
            L.warning(
                "Document you are trying to create already exists."
            )
            return None


    async def read(self, collection, obj_id):
        response = await self.MongoDBStorageService.get(
            collection, obj_id
        )
        return response


    async def update(self, collection, obj_id, document):
        original = await self.read(
            collection, obj_id
        )

        cre = self.MongoDBStorageService.upsertor(
            collection, original["_id"], original["_v"]
        )

        for key, value in zip(
            document.keys(), document.values()
        ):
            cre.set(key, value)

        try:
            await cre.execute()
            return "OK"

        except KeyError:
            return None


    async def delete(self, collection, obj_id):
        try:
            await self.MongoDBStorageService.delete(
                collection, obj_id
            )
            return True
        except KeyError:
            return False