Object Storage Functions

Object storage provides a distributed key-value store for managing data across your flows. Built on a replicated storage backend, it’s designed for reliable data persistence with configurable storage options.

What is Object Storage? Object storage is a key-value store optimized for distributed data management. Each bucket acts as a namespace for storing values by unique keys, with options for compression, replication, and automatic expiration.

Function List

obs-bucket-create

Create a new storage bucket

obs-bucket-list

List all storage buckets

obs-key-put

Store a value by key

obs-key-get

Retrieve a value by key

obs-key-list

List keys in a bucket

obs-bucket-create

Create a new object storage bucket to organize and store your data. Buckets provide isolated namespaces with configurable storage, replication, and lifecycle settings.

Parameters

bucket

string

required

Name of the bucket to create. Must be unique and can only contain alphanumeric characters, dashes, and underscores.Naming conventions:

Use lowercase with hyphens or underscores: customer-data, session_cache
Be descriptive: user-profiles not bucket1
Include purpose: temp-uploads, config-store

description

string

Optional description of the bucket’s purpose.

storage

string

default:"file"

Storage backend type. Options:

file - Persistent file storage (default, recommended for durability)
memory - In-memory storage (faster but not persistent across restarts)

num_replicas

integer

default:"1"

Number of data replicas to maintain across the cluster. Range: 1-5.Replication guidelines:

1 replica: Development or non-critical data
3 replicas: Production data requiring high availability
5 replicas: Mission-critical data with maximum redundancy

max_bytes

integer

default:"-1"

Maximum size in bytes for the bucket. Default is -1 (unlimited).Size planning:

Set limits for temporary storage to prevent unbounded growth
Leave unlimited for primary data stores
Consider setting based on available storage capacity

ttl

integer

Time-to-live in nanoseconds for keys in this bucket. Keys automatically expire after this duration. By default, keys do not expire.Common TTL values:

3600000000000 (1 hour) for session data
86400000000000 (24 hours) for temporary cache
604800000000000 (7 days) for short-term storage

compression

boolean

default:"false"

Enable stream compression to reduce storage space. Useful for text-heavy data or JSON objects.

metadata

object

Custom metadata key-value pairs to associate with the bucket.

placement

object

Cluster placement configuration for advanced deployment scenarios.Properties:

cluster (string): Target cluster name
tags (array): Array of placement tags

Response

{
  "status_code": 200,
  "body": {
    "message": "Bucket created successfully",
    "error": null
  }
}

Example Usage

{
  "function": "obs-bucket-create",
  "params": {
    "bucket": "user-profiles",
    "description": "Store user profile data",
    "storage": "file",
    "num_replicas": 3,
    "compression": true
  }
}

Common Use Cases

User Data Storage

Store user profiles, preferences, and state

Bucket: user-profiles
Storage: file (persistent)
Replicas: 3 (high availability)
Compression: enabled (for JSON data)

Session Management

Store temporary session data with auto-expiration

Bucket: user-sessions
Storage: memory (fast access)
TTL: 1 hour
Replicas: 1 (temporary data)

Configuration Cache

Store application configuration and settings

Bucket: config-store
Storage: file (persistent)
Replicas: 3 (important data)
No TTL (permanent storage)

Workflow State

Store state data for long-running workflows

Bucket: workflow-state
Storage: file (must persist)
Replicas: 3 (critical data)
Compression: enabled

obs-bucket-list

List all object storage buckets in your account with their metadata and statistics.

Parameters

This function takes no parameters.

Response

{
  "status_code": 200,
  "body": {
    "body": {
      "results_total": 3,
      "results": [
        {
          "name": "user-profiles",
          "description": "Store user profile data",
          "created": 1729000000000,
          "entryTotal": 1523,
          "metadata": {
            "environment": "production"
          }
        }
      ]
    },
    "metadata": {}
  }
}

Response fields:

results_total - Total number of buckets
results - Array of bucket objects:
- name - Bucket name
- description - Bucket description
- created - Creation timestamp (milliseconds)
- entryTotal - Number of keys in bucket
- metadata - Custom metadata

Example Usage

{
  "function": "obs-bucket-list",
  "params": {}
}

obs-key-put

Store a value in an object storage bucket by key. Values can be strings or JSON objects.

Parameters

bucket

string

required

Name of the bucket to store the value in. Bucket must already exist.

name

string

required

Unique key name for the value. This is the identifier used to retrieve the value.Key naming strategies:

User data: user:${userId}:profile
Hierarchical: config/app/database/connection
Timestamped: events/2025-10-16/event-123
UUID-based: Use UUIDs for guaranteed uniqueness

value

string | object

required

The value to store. Can be:

A string value (will be stored as-is)
A JSON object (will be automatically serialized)

Objects are automatically converted to JSON strings during storage. When retrieving, use the json flag to parse back to objects.

Response

{
  "status_code": 200,
  "body": {
    "message": "Value stored successfully",
    "error": null
  }
}

Example Usage

{
  "function": "obs-key-put",
  "params": {
    "bucket": "user-profiles",
    "name": "user:${$.user_id}",
    "value": {
      "email": "${$.email}",
      "name": "${$.name}",
      "created": "${$.timestamp}"
    }
  }
}

Common Patterns

User State Management

Store and update user state across flows

Store: User profile data, preferences, progress
Key pattern: user:{userId}:profile
Value: JSON object with user data
Retrieve: At start of user flows

Workflow Checkpoints

Save workflow state for resumption or recovery

Store: Current step, variables, progress
Key pattern: workflow:{workflowId}:state
Value: Object with complete state
Update: After each major step

Data Aggregation

Accumulate data over time

Store: Counters, metrics, aggregated data
Key pattern: metrics:{date}:{metric-name}
Value: Numeric or aggregated object
Update: Periodically or on events

obs-key-get

Retrieve a value from an object storage bucket by its key.

Parameters

bucket

string

required

Name of the bucket containing the key.

name

string

required

The key name of the value to retrieve.

json

boolean

default:"false"

Parse the stored value as JSON and return as an object. Use true when retrieving values that were stored as objects.When to use:

Set to true when the value is a JSON object
Set to false (or omit) when the value is a plain string

Response

{
  "status_code": 200,
  "body": {
    "body": {
      "name": "user:123",
      "value": {
        "email": "[email protected]",
        "name": "John Doe"
      }
    },
    "metadata": {
      "Bucket": "user-profiles"
    }
  }
}

Response fields:

body.body.name - The key name
body.body.value - The stored value (string or parsed object based on json parameter)
body.metadata.Bucket - The bucket name

Example Usage

{
  "function": "obs-key-get",
  "params": {
    "bucket": "user-profiles",
    "name": "user:${$.user_id}",
    "json": true
  }
}

Common Patterns

Load User Context

Retrieve user data at the start of a flow

Trigger: User action
Function: Get user profile with json=true
Use: Access user data in subsequent steps
Store in: Flow variables for easy access

Resume Workflow

Load saved state to continue a workflow

Trigger: Resume event
Function: Get workflow state with json=true
Parse: Extract step, variables, progress
Continue: From saved checkpoint

Configuration Loading

Load application settings and configuration

Trigger: App initialization
Function: Get config with json=true
Apply: Use settings throughout flow
Cache: In memory for performance

obs-key-list

List all keys in an object storage bucket with pagination support.

Parameters

bucket

string

required

Name of the bucket to list keys from.

last_sequence

integer

Starting sequence number for pagination. Returns keys after this sequence. Omit or use 0 for the first page.Use the modified value from the last result in the previous page to continue pagination.

limit

integer

Maximum number of keys to return per page. Controls result set size for pagination.

Response

{
  "status_code": 200,
  "body": {
    "body": {
      "results_total": 1523,
      "results": [
        {
          "name": "user:123",
          "buid": "abc123def456",
          "size": 2048,
          "chunks": 1,
          "modified": 1729080000000
        }
      ]
    },
    "metadata": {}
  }
}

Response fields:

results_total - Total number of keys in bucket
results - Array of key objects:
- name - Key name
- buid - Unique identifier
- size - Size in bytes
- chunks - Number of chunks
- modified - Last modified timestamp (milliseconds) - use for last_sequence in next page

Example Usage

{
  "function": "obs-key-list",
  "params": {
    "bucket": "user-profiles",
    "limit": 100
  }
}

Common Patterns

Batch Processing

Process all keys in a bucket sequentially

List keys with limit (e.g., 100)
Process each key in the batch
Use last result's modified for next page
Repeat until all keys processed

Data Audit

Audit or report on stored data

List all keys in bucket
Gather statistics: count, total size
Identify patterns in key names
Generate audit report

Cleanup Operations

Find and remove old or unused keys

List keys in bucket
Check modified timestamp
Delete keys older than threshold
Track cleanup metrics

Best Practices

Use Descriptive Keys

Use clear, hierarchical key naming patterns

Enable Replication

Use 3+ replicas for production data

Set TTL for Temp Data

Auto-expire temporary data to save storage

Use JSON Flag

Set json=true when retrieving objects

Compress Text Data

Enable compression for JSON/text buckets

Paginate Large Lists

Use limit and last_sequence for large datasets

Storage Types Comparison

Feature	File Storage	Memory Storage
Persistence	Survives restarts	Lost on restart
Speed	Moderate	Very fast
Capacity	Large	Limited by RAM
Use Case	Production data	Temporary cache
Cost	Storage-based	RAM-based

Next Steps

Stream Functions

Real-time event processing

Key-Value Storage

Fast key-based storage

Data Transformation

Transform and encode data

Functions Step

Using Functions in flows

Get Started

Flows

Agents

Account Management

Advanced

​Object Storage Functions

​Function List

obs-bucket-create

obs-bucket-list

obs-key-put

obs-key-get

obs-key-list

​obs-bucket-create

​Parameters

​Response

​Example Usage

​Common Use Cases

​obs-bucket-list

​Parameters

​Response

​Example Usage

​obs-key-put

​Parameters

​Response

​Example Usage

​Common Patterns

​obs-key-get

​Parameters

​Response

​Example Usage

​Common Patterns

​obs-key-list

​Parameters

​Response

​Example Usage

​Common Patterns

​Best Practices

Use Descriptive Keys

Enable Replication

Set TTL for Temp Data

Use JSON Flag

Compress Text Data

Paginate Large Lists

​Storage Types Comparison

​Next Steps

Stream Functions

Key-Value Storage

Data Transformation

Functions Step

Object Storage Functions

Function List

obs-bucket-create

Parameters

Response

Example Usage

Common Use Cases

obs-bucket-list

Parameters

Response

Example Usage

obs-key-put

Parameters

Response

Example Usage

Common Patterns

obs-key-get

Parameters

Response

Example Usage

Common Patterns

obs-key-list

Parameters

Response

Example Usage

Common Patterns

Best Practices

Storage Types Comparison

Next Steps