Developer's Guide to ZK Compression on Solana

From Theory to Practice

Solana's rapid growth has brought both opportunities and challenges, with state storage costs becoming a significant barrier for applications aiming to scale. With over 500 million accounts and roughly 1 million new accounts added daily, developers need efficient solutions for managing state growth. ZK Compression elegantly solves this by reducing state costs up to 5000x while preserving Solana's core security and composability guarantees.

In this comprehensive guide, we'll explore:

• How ZK Compression solves Solana's state growth challenges

• The technical architecture behind compressed accounts and state trees

• Practical examples of building with compressed accounts

• Best practices and limitations for production deployment

This article assumes knowledge of Solana’s Account model and basics of zero knowledge proofs.

Key Benefits

• ZK Compression reduces state costs by up to 5000x by storing data on Solana's cheaper ledger space

• Integration requires minimal changes to existing Solana development practices

• Supports atomic composability with regular Solana accounts and programs

• Ideal for applications needing many small accounts (tokens, game items, etc.)

• Use compressed tokens today with existing DeFi protocols through atomic decompress/compress operations

Table of Contents

1. Understanding the State Growth Problem

2. Introduction to ZK Compression

3. Technical Architecture

4. Project Tutorials

   1. Compressed Token Launch

   2. Jupiter Integration

   3. Compressed Blog dApp

5. Best Practices & Limitations

6. Conclusion & Additional Resources

Understanding the State Growth Problem

The Solana network faces increasing pressure on its state storage capacity. Let's break down the economics of traditional state storage:

// Traditional token account costs (at $150/SOL)
const traditionalCosts = {
    singleAccount: 0.00204428 SOL,  // ~$0.30
    thousandAccounts: 2.04428 SOL,  // ~$300
    millionAccounts: 2044.28 SOL,   // ~$300,000
};

This cost structure creates significant barriers for applications aiming to scale to millions of users. For example:

• A social network with 1M users: $300,000 in state costs

• A game with 100K players and 10 inventory slots each: $300,000

• A DeFi protocol with 1M token accounts: $300,000

Introduction to ZK Compression

ZK Compression is a new primitive built on Solana that enables you to build applications at scale. Developers and users can opt to compress their on-chain state, reducing state costs by orders of magnitude while preserving the security, performance, and composability of the Solana L1. The system achieves 5000x cheaper state costs through three core components:

• Compression: Only state roots (32-byte fingerprints of all compressed accounts) are stored in on-chain accounts. The underlying data is stored on the cheaper Solana ledger space instead of the more expensive account space.

• ZK (Zero-Knowledge): The protocol uses small zero-knowledge proofs (validity proofs) to ensure the integrity of the compressed state. These proofs maintain a constant 128-byte size regardless of how many accounts are being updated, read, or written in a single transaction.

• L1 Integration: Execution and data availability are both on Solana, preserving the performance and security guarantees of the L1. This is neither an L2 nor a validium - everything runs directly on Solana.

How ZK Compression Works

There are various working components which are working together, let’s learn about each component.

State Trees

The protocol stores compressed state in multiple state trees (a "forest" of trees). Each state tree:

• Is a binary Merkle tree that contains compressed account hashes as leaves

• Has a corresponding on-chain account storing only the final root hash and metadata

• Enables efficient cryptographic verification of any leaf's validity using the on-chain root

• Ensures each account hash is globally unique by including:

  • The state tree's public key (state tree hash)

  • The account's position (leafIndex)

  • The account's data hash

A Basic State Tree

Compressed Account Model

Compressed accounts are similar to regular Solana accounts but with key differences:

• Each compressed account can be identified by its hash

• Each write to a compressed account changes its hash

• An address can optionally be set as a permanent unique ID of the compressed account

• All compressed accounts are stored in sparse state trees

• Only the tree's state root is stored in the on-chain account space

• The raw data is stored in the ledger as transaction logs

Comparison between Regular and Compressed Token Accounts

Other than the differences listed above, both the token accounts are similar. Compressed Token Accounts are exactly like regular SPL token accounts from a developer's perspective - they can be transferred, delegated, frozen, and maintain the same security properties. They fully implement the SPL token standard, making them a drop-in replacement that's cheaper and more scalable.

Validity Proofs

• Uses Groth16, a renowned pairing-based zk-SNARK, for proof generation and verification

• Proofs maintain a constant 128-byte size regardless of how many accounts are being updated, read, or written in a single transaction

• Generated off-chain by Prover nodes and verified on-chain by the Light System Program

• Reduces Solana's computational burden while ensuring security

Program Architecture

• Light System Program (SySTEM1eSU2p4BGQfQpimFEWWSC1XDFeun3Nqzz3rT7): Core program handling compression operations and proof verification

• Account Compression Program (compr6CUsB5m2jS4Y3831ztGSTnDpnKJTKS95d64XVq): Implements state and address trees

• Compressed Token Program (cTokenmWW8bLPjZEBAUgYy3zKxQZW6VKi7bqNFEVv3m): A compressed token implementation built using ZK Compression that enforces SPL-compatible token layout

• All programs operate atomically on Solana's L1

Supporting Infrastructure

• Photon Indexer:

  • Tracks and indexes compression programs

  • Provides read state via ZK Compression RPC API

  • Caches current state for quick access

  • Can be run locally or accessed through RPC providers

• Forester Nodes:

  • Maintain state roots

  • Empty nullifier queues asynchronously

  • Handle state tree management

  • Operate permissionlessly - developers can run their own nodes or use network nodes

Forester Node

Transaction Flow

1. Client Operations

   • Gets compressed account state and Merkle proof from Photon RPC

   • Obtains validity proof from Prover node

   • Submits transaction with account state and proof

2. On-chain Protocol Execution The Light System Program:

   • Runs checks (sum check, etc.)

   • Verifies the validity proof

   • Nullifies old state

   • Appends new account hash to state tree

   • Updates state root

   • Emits new state to Solana ledger

This architecture enables efficient state compression while maintaining security through ZK proofs and full composability with other Solana programs through atomic operations. Developers can seamlessly interact with both compressed and regular accounts within the same transaction.

---

Project Tutorials

Compressed Token Launch

Let's look at a practical example of creating and using compressed tokens. This code demonstrates the core operations:

import { Rpc, createRpc } from "@lightprotocol/stateless.js";
import { createMint, mintTo, transfer } from "@lightprotocol/compressed-token";
import { Keypair, PublicKey } from "@solana/web3.js";
import bs58 from "bs58";
import "dotenv/config";

// Initialize RPC connection 
const RPC_ENDPOINT =
  "https://mainnet.helius-rpc.com/?api-key=" + process.env.HELIUS_API_KEY;
const connection: Rpc = createRpc(RPC_ENDPOINT, RPC_ENDPOINT);

const USER_KEYPAIR = Keypair.fromSecretKey(
  bs58.decode(process.env.WALLET_PRIVATE_KEY || "YOUR_PRIVATE_KEY")
);

const tokenAddress = Keypair.generate();
const receiver = Keypair.generate();

/// Create a compressed token mint
const { mint, transactionSignature } = await createMint(
  connection,
  USER_KEYPAIR,
  USER_KEYPAIR.publicKey,
  9, // Number of decimals
  tokenAddress // Optional
);

console.log(`create-mint  success! txId: ${transactionSignature}`);

/// Mint compressed tokens to the payer's account
const mintToTxId = await mintTo(
  connection,
  USER_KEYPAIR,
  mint,
  USER_KEYPAIR.publicKey, // Destination
  USER_KEYPAIR,
  1e10 // Amount
);

console.log(`mint-to  success! txId: ${mintToTxId}`);

/// Transfer compressed tokens
const transferTxId = await transfer(
  connection,
  USER_KEYPAIR,
  mint,
  7e8, // Amount
  USER_KEYPAIR, // Owner
  receiver.publicKey // Destination
);

console.log(`transfer  success! txId: ${transferTxId}`);

Jupiter Integration

One of ZK Compression's most powerful features is its atomic composability with existing Solana programs. Let's examine how to integrate compressed tokens with Jupiter:

In the following project, we will be writing a script to build a transaction that performs a Jupiter token swap and also compresses the output token atomically. We will batch the instructions for both actions in the same transaction.

Let’s import everything we require.

import {
  AddressLookupTableAccount,
  ComputeBudgetProgram,
  Keypair,
  PublicKey,
  TransactionMessage,
  VersionedTransaction,
} from "@solana/web3.js";
import bs58 from "bs58";
import "dotenv/config";
import { CompressedTokenProgram } from "@lightprotocol/compressed-token";
import axios from "axios";
import * as splToken from "@solana/spl-token";
import {
  Rpc,
  createRpc,
  sendAndConfirmTx,
  bn,
} from "@lightprotocol/stateless.js";
import BN from "bn.js";

Declaring the constants. Get a Helius API Key and add a Private Key to a wallet which has some USDC and Sol.

const RPC_ENDPOINT =
  "https://mainnet.helius-rpc.com/?api-key=" + process.env.HELIUS_API_KEY;
const PHOTON_ENDPOINT = RPC_ENDPOINT;
const connection: Rpc = createRpc(RPC_ENDPOINT, PHOTON_ENDPOINT);

const USER_KEYPAIR = Keypair.fromSecretKey(
  bs58.decode(process.env.WALLET_PRIVATE_KEY || "YOUR_PRIVATE_KEY")
);

export const tokenAddresses = {
  usdc: "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v",
  wif: "EKpQGSJtjMFqKZ9KQanSqYXRcF8fBopzLHYxdM65zcjm",
};

const JUPITER_ENDPOINT = `https://quote-api.jup.ag/v6`;
const SWAP_TOKEN_FROM = tokenAddresses.usdc;
const SWAP_TOKEN_TO = tokenAddresses.wif; 
const SLIPPAGE_BPS = 100;

We would need a few helper functions, let’s declare them. These helper functions would help us to deserialize transactions and encode them.

///// helper functions

async function deserializeTransaction(swapTransaction: string) {
  const swapTransactionBuf = Buffer.from(swapTransaction, "base64");
  const tx = VersionedTransaction.deserialize(swapTransactionBuf);
  const addressLookupTableAccounts = await Promise.all(
    tx.message.addressTableLookups.map(async (lookup) => {
      return new AddressLookupTableAccount({
        key: lookup.accountKey,
        state: AddressLookupTableAccount.deserialize(
          await connection
            .getAccountInfo(lookup.accountKey)
            .then((res) => res!.data)
        ),
      });
    })
  );
  return { tx, addressLookupTableAccounts };
}

function encodeBase64Bytes(bytes) {
  return btoa(
    bytes.reduce((acc, current) => acc + String.fromCharCode(current), "")
  );
}

We have all the things we require, now we would be using the Jupiter API to fetch the swapping instructions and the Light SDK will help us get the compressing instructions.

async function swapAndCompress() {
  console.log("Let's do it all in one transaction");
  let mergedMessage: TransactionMessage | null = null;

  /// Swap

  const amount = new BN(1_000_000); // 1 USDC

  const data = await axios.get(
    `${JUPITER_ENDPOINT}/quote?inputMint=${SWAP_TOKEN_FROM}&outputMint=${SWAP_TOKEN_TO}&amount=${amount}&slippageBps=${SLIPPAGE_BPS}&maxAccounts=8`
  );
  const quoteResponse = data.data;
  const userPublicKey = USER_KEYPAIR.publicKey.toBase58();
  const swapInstructions = await axios.post(`${JUPITER_ENDPOINT}/swap`, {
    quoteResponse,
    userPublicKey,
    wrapAndUnwrapSol: false,
    dynamicComputeUnitLimit: true,
    prioritizationFeeLamports: "auto",
  });

  const swapTransaction = swapInstructions.data.swapTransaction;
  console.log("got swapInstructions");

  /// compress wif token

  const sourceTokenAccount = await splToken.getOrCreateAssociatedTokenAccount(
    // @ts-ignore
    connection,
    USER_KEYPAIR,
    new PublicKey(tokenAddresses.wif),
    USER_KEYPAIR.publicKey
  );

  const ix = await CompressedTokenProgram.compress({
    payer: USER_KEYPAIR.publicKey,
    owner: USER_KEYPAIR.publicKey,
    source: new PublicKey(sourceTokenAccount.address.toBase58()),
    toAddress: USER_KEYPAIR.publicKey,
    mint: new PublicKey(tokenAddresses.wif),
    amount: bn(10),
  });

  console.log("got compressInstructions");

  //// merging instructions

  const { tx, addressLookupTableAccounts } = await deserializeTransaction(
    swapTransaction
  );

  const message = TransactionMessage.decompile(tx.message, {
    addressLookupTableAccounts,
  });

  mergedMessage = message;

  mergedMessage.instructions[0] = ComputeBudgetProgram.setComputeUnitLimit({
    units: 400_000,
  });

  mergedMessage.instructions[1] = ComputeBudgetProgram.setComputeUnitPrice({
    microLamports: 30_000,
  });

  /// add compress instruction

  mergedMessage.instructions.push(ix);

  /// forming transaction

  const transaction = new VersionedTransaction(
    mergedMessage.compileToV0Message(addressLookupTableAccounts)
  );

  transaction.sign([USER_KEYPAIR]);

  /// verify the encoded transaction in https://explorer.solana.com/tx/inspector

  console.log(encodeBase64Bytes(transaction.serialize()));

  /// send it

  const txId = await sendAndConfirmTx(connection, transaction);

  console.log("Transaction Signature:", txId);
}

This pattern allows users to seamlessly interact with existing DeFi protocols while benefiting from compressed token storage. Here is an additional code snippet in case you require to decompress your SPL token.

async function decompress() {
  const sourceTokenAccount = await splToken.getOrCreateAssociatedTokenAccount(
    // @ts-ignore
    connection,
    USER_KEYPAIR,
    new PublicKey(tokenAddresses.usdc),
    USER_KEYPAIR.publicKey
  );

  const compressedTokenAccounts =
    await connection.getCompressedTokenAccountsByOwner(USER_KEYPAIR.publicKey, {
      mint: new PublicKey(tokenAddresses.usdc),
    });

  const [inputAccounts] = selectMinCompressedTokenAccountsForTransfer(
    compressedTokenAccounts.items,
    bn(10)
  );

  const proof = await connection.getValidityProof(
    inputAccounts.map((account) => bn(account.compressedAccount.hash))
  );

  const amount = new BN(10);

  console.log("Amount with decimals:", amount.toString());

  const decompressIx = await CompressedTokenProgram.decompress({
    payer: USER_KEYPAIR.publicKey,
    inputCompressedTokenAccounts: inputAccounts,
    toAddress: sourceTokenAccount.address,
    amount,
    recentInputStateRootIndices: proof.rootIndices,
    recentValidityProof: proof.compressedProof,
  });

  const { blockhash } = await connection.getLatestBlockhash();

  const tx = buildAndSignTx(
    [
      ComputeBudgetProgram.setComputeUnitLimit({ units: 1_200_000 }),
      decompressIx,
    ],
    USER_KEYPAIR,
    blockhash,
    []
  );

  console.log("Sending TX");

  /// Confirm

  const txId = await sendAndConfirmTx(connection, tx);
  console.log("Transaction Signature:", txId);
}

Compressed Blog dApp

Let's examine a practical example of using ZK Compression by building a blog platform. This example demonstrates how compression can be used for content storage while maintaining security and composability.

Prerequisites

1. Install Rust, Solana CLI and Anchor CLI from here.

2. Install Light CLI

Start a new project using

light init myjournal

Project Structure

use anchor_lang::prelude::*;
use light_sdk::{
    compressed_account::LightAccount,
    light_account,
    light_accounts,
    light_program,
    merkle_context::PackedAddressMerkleContext,
};

declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");

Core Components:

1. Program Module

#[light_program]
#[program]
pub mod myjournal {
    use super::*;

    // Program instructions...
}

The #[light_program] macro is key here as it:

• Enables ZK compression for the program

• Sets up necessary interfaces for compressed state handling

• Manages state transitions and proof verification

2. Account State

#[light_account]
#[derive(Clone, Debug, Default)]
pub struct BlogEntryState {
    #[truncate]
    pub owner: Pubkey,
    pub title: String,
    pub message: String,
}

This structure defines our compressed blog post where:

• #[light_account] enables compression for the struct

• #[truncate] optimizes the Pubkey storage

• Content is stored in the ledger, with only the root hash on-chain

3. Instructions

a. Create Blog Entry:

pub fn create_entry<'info>(
    ctx: LightContext<'_, '_, '_, 'info, CreateEntry<'info>>,
    title: String,
    message: String
) -> Result<()> {
    msg!("Creating blog entry");
    
    // Set entry data
    ctx.light_accounts.blog_entry.owner = ctx.accounts.signer.key();
    ctx.light_accounts.blog_entry.title = title;
    ctx.light_accounts.blog_entry.message = message;

    Ok(())
}

b. Update Blog Entry:

pub fn update_entry<'info>(
    ctx: LightContext<'_, '_, '_, 'info, UpdateEntry<'info>>,
    title: String,
    message: String
) -> Result<()> {
    msg!("Updating blog entry");
    // Verify ownership
    if ctx.light_accounts.blog_entry.owner != ctx.accounts.signer.key() {
        return Err(CustomError::Unauthorized.into());
    }

    ctx.light_accounts.blog_entry.title = title;
    ctx.light_accounts.blog_entry.message = message;

    Ok(())
}

4. Instruction Contexts

#[light_accounts]
#[instruction(title: String, message: String)]
pub struct CreateEntry<'info> {
    #[account(mut)]
    #[fee_payer]
    pub signer: Signer<'info>,
    #[self_program]
    pub self_program: Program<'info, crate::program::Myjournal>,
    #[authority]
    pub cpi_signer: AccountInfo<'info>,
    #[light_account(init, seeds = [b"BLOG", title.as_bytes(), signer.key().as_ref()])]
    pub blog_entry: LightAccount<BlogEntryState>,
}

This context structure defines:

• Who's paying for the transaction (fee_payer)

• Program identification (self_program)

• Authority for compression (authority)

• The compressed blog entry account with PDA seeds

Here is the GitHub repo for the entire code.

Cost Benefits

Traditional blog post storage:

const traditionalPostCost = {
    rentExempt: 0.00204428 SOL,  // Per post
    millionPosts: 2044.28 SOL,   // ~$300,000 at $150/SOL
};

Compressed blog post storage:

const compressedPostCost = {
    stateRoot: 0.00000408 SOL,   // Per post
    millionPosts: 4.08 SOL,      // ~$612 at $150/SOL
};

This implementation provides:

• Nearly 500x cost reduction for content storage

• Full content ownership verification

• Atomic updates with proof verification

• Seamless integration with other Solana programs

The Light Protocol macros handle all the complexity of:

• Proof generation and verification

• State compression and decompression

• Merkle tree management

• PDA derivation

• Transaction construction

This allows developers to focus on application logic while benefiting from the massive cost savings of ZK Compression.

Best Practices & Limitations

When building with ZK Compression, keep these guidelines in mind:

Best Practices

1. Transaction Size Management

   • Reserve 128 bytes for validity proof, max Solana Txn size limit is 1232 bytes.

   • Use lookup tables for multiple account references, you can even create your own lookup tables using Light’s SDK.

   • Batch operations when possible

2. Account Design

   • Group related data into single compressed accounts

   • Use appropriate data types to minimize storage

   • Consider update frequency when choosing compression

3. Performance Optimization

   • Batch state updates

   • Monitor compute unit usage

Limitations

1. Transaction Size

   • 1232 byte limit includes proof

   • May require splitting large operations in multiple transactions

2. Compute Units

   • ~100k CU for proof verification

   • ~6k CU per compressed account operation

3. Update Frequency

   • Better for cold storage than hot state

   • Consider decompression for frequently updated accounts

Conclusion

ZK Compression represents a significant advancement in Solana's scalability story. By dramatically reducing state costs while maintaining security and composability, it enables a new generation of applications that can scale to millions of users.

Whether you're building a token project, a game, or a social platform, ZK Compression provides the tools needed to scale efficiently on Solana. The combination of compressed state, zero-knowledge proofs, and atomic composability creates a powerful primitive for the future of decentralized applications.

For developers looking to get started:

1. Experiment with compressed tokens

2. Understand the transaction lifecycle

3. Plan for account structure and scaling

4. Consider compression in your application architecture

The examples and patterns shown here provide a foundation for building with ZK Compression, but the possibilities are limited only by your imagination.

Additional Resources

• ZKCompression.com

• Helius Endpoints for ZK Compression Indexer

• Ideas to build

Happy Hacking!!