Quick Start¶
This quick start guide will walk you through the process of installing and using the ReductStore Python client SDK to interact with a ReductStore instance.
Installing the SDK¶
To install the ReductStore SDK, you will need to have Python 3.7 or higher installed on your machine. Once Python is
installed, you can use the pip package manager to install the reduct-py package:
pip install reduct-py
Running ReductStore as a Docker Container¶
If you don't already have a ReductStore instance running, you can easily spin one up as a Docker container. To do so, run the following command:
docker run -p 8383:8383 reduct/store:latest
This will start a ReductStore instance listening on port 8383 of your local machine.
Using the SDK¶
Now that you have the reduct-py SDK installed and a ReductStore instance running, you can start using the SDK to
interact with the ReductStore service.
Here is an example of using the Python SDK to perform a few different operations on a bucket:
from pathlib import Path
from time import time_ns
from reduct import Client, BucketSettings, QuotaType
CURRENT_FILE = Path(__file__)
async def main():
# --8<-- [start:createclient]
# Create a ReductStore client
async with Client("http://localhost:8383") as client:
# --8<-- [end:createclient]
# --8<-- [start:createbucket]
# Get or create a bucket with 1Gb quota
bucket = await client.create_bucket(
"my-bucket",
BucketSettings(quota_type=QuotaType.FIFO, quota_size=1_000_000_000),
exist_ok=True,
)
# --8<-- [end:createbucket]
# --8<-- [start:writedata]
# The simplest case. Write some data with the current timestamp
await bucket.write("entry-1", b"Hello, World!")
# --8<-- [end:writedata]
# --8<-- [start:uploadfile]
# More complex case. Upload a file in chunks with a custom timestamp unix timestamp in microseconds
async def file_reader():
"""Read the current example in chunks of 50 bytes"""
with open(CURRENT_FILE, "rb") as file:
while True:
data = file.read(50) # Read in chunks of 50 bytes
if not data:
break
yield data
ts = int(time_ns() / 1000)
await bucket.write(
"entry-1",
file_reader(),
timestamp=ts,
content_length=CURRENT_FILE.stat().st_size,
)
# --8<-- [end:uploadfile]
# --8<-- [start:readdata]
# The simplest case. Read the data by a certain ts
async with bucket.read("entry-1", timestamp=ts) as record:
print(f"Record timestamp: {record.timestamp}")
print(f"Record size: {record.size}")
print(await record.read_all())
# --8<-- [end:readdata]
# --8<-- [start:iteraterecords]
# More complex case. Iterate over all records in the entry and read them in chunks
async for record in bucket.query("entry-1"):
print(f"Record timestamp: {record.timestamp}")
print(f"Record size: {record.size}")
async for chunk in record.read(50):
print(chunk)
# --8<-- [end:iteraterecords]
if __name__ == "__main__":
import asyncio
asyncio.run(main())
Let's break down what this example is doing.
Creating a Client¶
To create a ReductStore client, you can use the Client class from the reduct module. Pass the URL of the ReductStore
instance you want to connect to as an argument to the Client constructor. To reuse the same HTTP session and improve
performance, you can use the context manager:
# Create a ReductStore client
async with Client("http://localhost:8383") as client:
Creating a Bucket¶
To get or create a bucket, you can use the create_bucket method on a Client instance. Pass the name of the bucket
you
want to get or create as an argument, along with a BucketSettings object to specify the desired quota for the bucket.
Set the exist_ok argument to True to create the bucket if it does not exist:
# Get or create a bucket with 1Gb quota
bucket = await client.create_bucket(
"my-bucket",
BucketSettings(quota_type=QuotaType.FIFO, quota_size=1_000_000_000),
exist_ok=True,
)
Writing data to a Bucket¶
To write data to an entry in a bucket with the current timestamp, you can use the write method on a Bucket instance.
Pass the name of the entry you want to write to as an argument, along with the data you want to write. The write
method
will automatically use the current timestamp when writing the data to the entry:
# The simplest case. Write some data with the current timestamp
await bucket.write("entry-1", b"Hello, World!")
This is the simplest case for writing data to an entry in a bucket using the SDK. The write method also allows you to specify a custom timestamp. Let's look at an example of uploading data in chunks with a custom timestamp:
To upload a file in chunks to an entry in a bucket, pass the name of
the entry you want to write to as an argument, along with a generator function that yields the file data in chunks.
Specify the total size of the file with the content_length argument, and specify a custom timestamp with
the timestamp
argument:
# More complex case. Upload a file in chunks with a custom timestamp unix timestamp in microseconds
async def file_reader():
"""Read the current example in chunks of 50 bytes"""
with open(CURRENT_FILE, "rb") as file:
while True:
data = file.read(50) # Read in chunks of 50 bytes
if not data:
break
yield data
ts = int(time_ns() / 1000)
await bucket.write(
"entry-1",
file_reader(),
timestamp=ts,
content_length=CURRENT_FILE.stat().st_size,
)
This code snippet shows how to use the write method to upload a file in chunks to an entry in a bucket. The generator
function file_reader reads the file in chunks of 50 bytes and yields the data. The write method reads the data from
the
generator and writes it to the entry in the bucket.
Reading data from a Bucket¶
To read data from an entry in a bucket, you can use the read method on a Bucket instance. Pass the name of the entry
you
want to read from and the timestamp of the specific record you want to read as arguments. Use the async with statement
to open the record, and then use the read_all method to read all of the data in the record:
# The simplest case. Read the data by a certain ts
async with bucket.read("entry-1", timestamp=ts) as record:
print(f"Record timestamp: {record.timestamp}")
print(f"Record size: {record.size}")
print(await record.read_all())
To iterate over all records in an entry, you can use the query method on a Bucket instance. Pass the name of the entry
you want to iterate over as an argument. Use a for loop to iterate over the records, and use the read method on each
record to read the data in chunks:
# More complex case. Iterate over all records in the entry and read them in chunks
async for record in bucket.query("entry-1"):
print(f"Record timestamp: {record.timestamp}")
print(f"Record size: {record.size}")
async for chunk in record.read(50):
print(chunk)
The query method returns an async iterator to the records in the entry. By default, the method returns all records
in the entry. However, you can use the start and stop arguments to specify a time interval for the records you want
to
retrieve. The start argument specifies the beginning of the time interval, and the stop argument specifies the end
of
the time interval. Both arguments are timestamps in UNIX microseconds.
start_ts = 1609459200000000 # January 1, 2021 at 00:00:00
stop_ts = 1609827200000000 # January 31, 2021 at 23:59:59
async for record in bucket.query("entry-1", start=start_ts, stop=stop_ts):
# Process the record
Next Steps¶
You can find more detailed documentation and examples in the Reference API section. You can also refer to the ReductStore HTTP API documentation for a complete reference of the available API calls.