

Introduction to RedisAI

What is RedisAI?

RedisAI is a Redis module for executing Deep Learning/Machine Learning models and managing their data. Its purpose is being a "workhorse" for model serving, by providing out-of-the-box support for popular DL/ML frameworks and unparalleled performance. RedisAI both simplifies the deployment and serving of graphs by leveraging on Redis' production-proven infrastructure, as well as maximizes computation throughput by adhering to the principle of data locality.

This introduction is intended to present the core concepts it uses and the functionality it provides.

Prerequisites

Before diving into RedisAI please make sure that you are familiar with the basic concepts of machine learning and Redis.

In broad strokes, RedisAI's looks as follows:

+-----------------------------------------------------------------------------+
| SERVER                                                                      |
| +-------------------------------------------------------------------------+ |
| | REDIS                              +----------+                         | |
| | +----------+   +-----------------+ | Commands | +---------------------+ | |
| | | KEYSPACE |   |  REDISAI          +----+-----+                       | | |
| | |          |   |                        ^                             | | |
| | +----------+   |  Data Structures       |             DL/ML Backends  | | |
| | |          |   |  +--------+            |             +-------------+ | | |
| | | mytensor +----->+ Tensor +<--+        |         +-->+ TensorFlow  | | | |
| | |          |   |  +--------+   |        |         |   +-------------+ | | |
| | +----------+   |               |        |         |                 | | | |
| | |          |   |  +--------+   |        v         |   +-------------+ | | |
| | | mymodel  +----->+ Model  +---+   +----+-----+   +-->+   PyTorch   | | | |
| | |          |   |  +--------+   |   |          |   |   +-------------+ | | |
| | +----------+   |               +-->+  Engine  +<--+                   | | |
| | |          |   |  +--------+   |   |          |   |   +-------------+ | | |
| | | myscript +----->+ Script +---+   +----+-----+   +-->+ ONNXRuntime | | | |
| | |          |   |  +--------+   |        ^         |   +-------------+ | | |
| | +----------+   |               |        |         |                   | | |
| | |          |   |  +--------+   |        |         |   +-------------+ | | |
| | | mydag    +----->+  DAG   +---+        |         +-->+     ...     | | | |
| | |          |   |  +--------+            |             +-------------+ | | |
| | +----------+ ^ +------------------------|-----------------------------+ | |
| +--------------|--------------------------|-------------------------------+ |
|                v                          v                                 |
| +--------------+-----------------+   +------------------------------------+ |
| | RAM                            |   | DEVICES                            | |
| |                                |   | +-----+  +-----+  +-----+  +-----+ | |
| | 00101010 00101010 00101010 ... |   | | CPU |  | GPU |  | TPU |  | ... | | |
| |                                |   | +-----+  +-----+  +-----+  +-----+ | |
| +--------------------------------+   +------------------------------------+ |
+-----------------------------------------------------------------------------+

What is Redis?

Redis is an ...

https://redis.io

... open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster.

Redis is also...

An acronym for REmote DIctionary Server that's pronounced "red" like the color, then "iss".

Further reference

More about Redis:

• Home page
• Interactive tutorial
• Repository

Basically, Redis is many things, but one of the things it is best known for is being a Key-Value server. That means that the data in Redis is represented as keys - unique string identifiers - and their respective values.

For example, this shows the basic use of the native Redis String data structure using the SET and GET commands:

redis> SET mykey somevalue
OK
redis> GET mykey
"somevalue"

What is a Redis Module?

A Redis module is a shared library that can be loaded by the Redis server during runtime. Modules are useful for extending Redis with new data structures and logic. A Redis module leverages the server's infrastructure. RedisAI is implemented as a module, which allows it direct access to the data managed by the server.

Further reference

You can learn more about Redis modules at:

• Redis Modules API
• Modules published at redis.io

Why RedisAI?

RedisAI bundles together best-of-breed technologies for delivering stable and performant graph serving. Every DL/ML framework ships with a backend for executing the graphs developed by it, and the common practice for serving these is building a simple server.

RedisAI aims to be that server, saving you from the need of installing the backend you're using and developing a server for it. By itself that does not justify RedisAI's existence so there's more to it. Because RedisAI is implemented as a Redis module it automatically benefits from the server's capabilities: be it Redis' native data types, its robust eco-system of clients, high-availability, persistence, clustering, and Enterprise support.

Because Redis is an in-memory data structure server RedisAI uses it for storing all of its data. The main data type supported by RedisAI is the Tensor that is the standard representation of data in the DL/ML domain. Because tensors are stored memory space of the Redis server they are readily accessible to any of RedisAI's backend libraries at minimal latency.

The locality of data, which is tensor data in adjacency to DL/ML models backends, allows RedisAI to provide optimal performance when serving models. It also makes it a perfect choice for deploying DL/ML models in production and allowing them to be used by any application.

Furthermore, RedisAI is also an optimal testbed for models as it allows the parallel execution of multiple graphs and, in future versions, assessing their respective performance in real-time.

Data Structures

RedisAI provides the following data structures:

• Tensor: represents an n-dimensional array of values
• Model: represents a frozen graph by one of the supported DL/ML framework backends
• Script: represents a TorchScript

DL/ML Backends

RedisAI supports the following DL/ML identifiers and respective backend libraries:

• TF: the TensorFlow backend
• TFLITE: The TensorFlow Lite backend
• TORCH: The PyTorch backend
• ONNX: ONNXRuntime backend

Backend libraries are dynamically loaded as needed, but can also be loaded during booting or at runtime.

Further reference

Refer to these pages for more information on loading backends:

• AI.CONFIG command
• Backend configuration

Getting Started

The easiest way to get a standalone Redis server with RedisAI bootstrapped locally is to use the official RedisAI Docker container image:

docker run -d --name redisai -p 6379:6379 redisai/redisai:latest

Further reference

For more information on installing RedisAI refer to the Quickstart page.

Once the server is up, you can connect to it using any Redis client. Better yet, some languages already have client implementations for RedisAI - the list can be found at the Clients page. RedisAI clients wrap the core API and simplify the interaction with the module.

We'll begin by using the official redis-cli Redis client. If you have it locally installed feel free to use that, but it is also available from the container:

docker exec -it redisai redis-cli

Using RedisAI Tensors

A Tensor is an n-dimensional array and is the standard vehicle for DL/ML data. RedisAI adds to Redis a Tensor data structure that implements the tensor type. Like any datum in Redis, RedisAI's Tensors are identified by key names.

Creating new RedisAI tensors is done with the AI.TENSORSET command. For example, consider the tensor: $\begin{equation*} tA = \begin{bmatrix} 2 \\ 3 \end{bmatrix} \end{equation*}$.

We can create the RedisAI Tensor with the key name 'tA' with the following command:

AI.TENSORSET tA FLOAT 2 VALUES 2 3

Copy the command to your cli and hit the <ENTER> on your keyboard to execute it. It should look as follows:

Example: setting a tensor

$ docker exec -it redisgears redis-cli
127.0.0.1:6379> AI.TENSORSET tA FLOAT 2 VALUES 2 3
OK

The reply 'OK' means that the operation was successful. We've called the AI.TENSORSET command to set the key named 'tA' with the tensor's data, but the name could have been any string value. The FLOAT argument specifies the type of values that the tensor stores, and in this case a double-precision floating-point. After the type argument comes the tensor's shape as a list of its dimensions, or just a single dimension of 2.

The VALUES argument tells RedisAI that the tensor's data will be given as a sequence of numeric values and in this case the numbers 2 and 3. This is useful for development purposes and creating small tensors, however for practical purposes the AI.TENSORSET command also supports importing data in binary format.

The Redis key 'tA' now stores a RedisAI Tensor. We can verify that using standard Redis commands such as EXISTS and TYPE:

Example: tensor key existence and type

127.0.0.1:6379> EXISTS tA
(integer) 1
127.0.0.1:6379> TYPE tA
AI_TENSOR

Using AI.TENSORSET with the same key name, as long as it already stores a RedisAI Tensor, will overwrite the existing data with the new. To delete a RedisAI tensor, use the Redis DEL command.

RedisAI Tensors are used as inputs and outputs in the execution of models and scripts. For reading the data from a RedisAI Tensor value there is the AI.TENSORGET command:

Example: getting a tensor key

127.0.0.1:6379> AI.TENSORGET tA VALUES
1) INT8
2) 1) (integer) 2
3) 1) (integer) 2
   1) (integer) 3

Loading Models

A Model is a Deep Learning or Machine Learning frozen graph that was generated by some framework. The RedisAI Model data structure represents a DL/ML model that is stored in the database and can be run.

Models, like any other Redis and RedisAI data structures, are identified by keys. A Model's key is created using the AI.MODELSET command and requires the graph payload serialized as protobuf for input.

In our examples, we'll use one of the graphs that RedisAI uses in its tests, namely 'graph.pb', which can be downloaded from here.

Downloading 'graph.pb'

Use a web browser or the command line to download 'graph.pb':

wget https://github.com/RedisAI/RedisAI/raw/master/test/test_data/graph.pb

redis-cli doesn't provide a way to read files' contents, so to load the model with it we'll use the command line and output pipes:

cat graph.pb | docker exec -i redisai redis-cli -x \
               AI.MODELSET mymodel TF CPU INPUTS a b OUTPUTS c

Example: loading a model from command line

$ cat graph.pb | docker exec -i redisai redis-cli -x \
           AI.MODELSET mymodel TF CPU INPUTS a b OUTPUTS c
OK

Use a client in your language of choice

For practical purposes, you are encouraged to use a programmatic Redis or RedisAI client in the language of your choice for interacting with RedisAI. Refer to the following pages for further information:

• Redis clients page
• RedisAI clients page

Like most commands, AI.MODELSET's first argument is a key's name, which is 'mymodel' in the example. The next two arguments are the model's DL/ML backend and the device it will be executed on. 'graph.pb' in the example is a TensorFlow graph and is denoted by TF argument. The model will be executed on the CPU as instructed by the CPU argument.

TensorFlow models also require declaring the names of their inputs and outputs. The inputs for 'graph.pb' are called 'a' and 'b', whereas its single output is called 'c'. These names are provided as additional arguments after the 'INPUTS' and 'OUTPUTS' arguments, respectively.

Running Models

Once a RedisAI Model key has been set with AI.MODELSET it can be run with any Tensor keys from the database as its input. The model's output, after it was executed, is stored in RedisAI Tensors as well.

The model stored at 'mymodel' expects two input tensors so we'll use the previously-create 'tA' and create another input tensor, $\begin{equation*} tB = \begin{bmatrix} 3 \\ 5 \end{bmatrix} \end{equation*}$, with the following command:

AI.TENSORSET tB FLOAT 2 VALUES 3 5

The model can now be run with the AI.MODELRUN command as follows:

AI.MODELRUN mymodel INPUTS tA tB OUTPUTS tResult

Example: running a model

127.0.0.1:6379> AI.TENSORSET tA FLOAT 2 VALUES 2 3
OK
127.0.0.1:6379> AI.TENSORSET tB FLOAT 2 VALUES 3 5
OK
127.0.0.1:6379> AI.MODELRUN mymodel INPUTS tA tB OUTPUTS tModel
OK

The first argument to AI.MODELRUN is the name of the key at which the RedisAI Model is stored. The names of RedisAI Tensor keys that follow the INPUTS argument are used as input for the model. Similarly, following the OUTPUT argument are the key names of RedisAI Tensors that the model outputs.

The inputs for the example are the tensors stored under the 'tA' and 'tB' keys. Once the model's run had finished, a new RedisAI Tensor key called 'tResult' is created and stores the model's output.

Example: fetching the model's output

127.0.0.1:6379> AI.TENSORGET tModel VALUES
1) FLOAT
2) 1) (integer) 2
3) 1) "6"
   2) "15"

The model we've imported from 'graph.pb' takes two input tensors as input and outputs a tensor that is the product of multiplying them. In the case of the example above it looks like this:

$$\begin {equation*} tModel = tA \times tB = \begin{bmatrix} 2 \\ 3 \end{bmatrix} \times \begin{bmatrix} 3 \\ 5 \end{bmatrix} = \begin{bmatrix} 2 \times 3 \\ 3 \times 5 \end{bmatrix} = \begin{bmatrix} 6 \\ 15 \end{bmatrix} \end {equation*}$$

Model Management

The AI.MODELGET command can be used for retrieving information about a model and its serialized blob. The AI.INFO command shows runtime statistics about the model's runs. Lastly, RedisAI Model keys can be deleted with the AI.MODELDEL command.

Scripting

RedisAI makes it possible to run TorchScript with the PyTorch backend. Scripts are useful for performing pre- and post-processing operations on tensors.

The RedisAI Script data structure is managed via a set of dedicated commands, similarly to the models. A RedisAI Script key is:

• Created with the AI.SCRIPTSET command
• Run with the AI.SCRIPTRUN command
• Deleted with the AI.SCRIPTSEL command

We can create a RedisAI Script that performs the same computation as the 'graph.pb' model. The script can look like this:

def multiply(a, b):
    return a * b

Assuming that the script is stored in the 'myscript.py' file it can be uploaded via command line and the AI.SCRIPTSET command as follows:

cat myscript.py | docker exec -i redisai redis-cli -x AI.SCRIPTSET myscript CPU

This will store the PyTorch Script from 'myscript.py' under the 'myscript' key and will associate it with the CPU device for execution. Once loaded, the script can be run with the following:

AI.SCRIPTRUN myscript multiply INPUTS tA tB OUTPUTS tScript

Example: running a script

127.0.0.1:6379> AI.TENSORSET tA FLOAT 2 VALUES 2 3
OK
127.0.0.1:6379> AI.TENSORSET tB FLOAT 2 VALUES 3 5
OK
127.0.0.1:6379> AI.SCRIPTRUN myscript multiply INPUTS tA tB OUTPUTS tScript
OK
127.0.0.1:6379> AI.TENSORGET tScript VALUES
1) FLOAT
2) 1) (integer) 2
3) 1) "6"
   2) "15"

Where Next?

This covers the basics of RedisAI's data structures and capabilities. For more information refer to:

• The Commands page provides a reference of the RedisAI API
• The Clients page lists RedisAI clients in several programming languages
• The Examples page showcases several examples that can be extended