Storage Bridge JS

Warning: This is pre-release software!

The @textile/near-storage library provides zero-config Typescript/Javascript SDKs that make it easy to store data on the Filecoin network from any NEAR-based dApp. @textile/near-storage should feel comfortable to developers already familiar with NEAR Javascript libraries. This chain-specific SDK provides a small but powerful API surfaces that integrates nicely with existing NEAR development best practices. Simply import the library, deposit some funds, and you are ready to start submitting data to be stored on the Filecoin network.

For a great overview of the Filecoin Bridge system, check out this introductory video.

Install

npm i @textile/near-storage

Usage

import { connect, WalletConnection } from "near-api-js";
import { init, requestSignIn } from "@textile/near-storage";

// Defaults to Testnet: https://near.github.io/near-api-js/modules/browserconnect.html
const near = await connect({ ... });

// Need to access wallet
const wallet = new WalletConnection(near, null);
await requestSignIn(wallet);

const storage = init(wallet.account());

const blob = new Blob(["Hello, world!"], { type: "text/plain" });
const file = new File([blob], "welcome.txt", {
  type: "text/plain",
  lastModified: new Date().getTime(),
});

await storage.addDeposit();

const { id, cid } = await storage.store(file);

const { request, deals } = await storage.status(id)
console.log(request.status_code)
console.log([...deals])

await wallet.signOut();

API

Full library documentation (TypeDocs), available on GitHub Pages!

The main @textile/near-storage entry point exposes an initialization function that takes a NEAR Account object, and returns a Storage object with a minimal CoreAPI interface. The initialization function can optionally take information about a known Filecoin storage provider, otherwise, a provider is automatically selected:

import { connect, WalletConnection } from "near-api-js";
import { init, requestSignIn } from "@textile/near-storage";

// See https://github.com/textileio/storage-js-basic-demo/ for a basic demo

// Defaults to Testnet: https://near.github.io/near-api-js/modules/browserconnect.html
const near = await connect({ ... });

// Need to access wallet
const wallet = new WalletConnection(near, null);

// Sign-in and authorize the @textile/near-storage smart contract
await requestSignIn(wallet)

// Initialize the storage object, and you're ready to go
const storage = await init(wallet.account());

Network selection

By default, @textile/near-storage will begin using the NEAR Testnet. To change networks, review the network specifications here.

Create session

The core storage API revolves around two key concepts: deposits and storage. Leaving a deposit provides a degree of Sybil resistance, such that users looking to store data on Filecoin via the provider must first deposit funds proportional to the length of time they'd like to continue storing data (for Testnet, the default timeout is ~10 minutes). To store data, a minimum (default) deposit must be left with a provider:

const deposit = await storage.addDeposit();
console.log(deposit);

A deposit is generally valid for about 10 minutes (based on blocks) on NEAR. The session duration is a function of the amount of funds deposited. Currently, on NEAR this is about 0.25 NEAR for a 10 minute session. After funds expire, they can be released by the user or any other party interacting with the SDK's smart contract (such as the provider itself). This provides a means to release funds after a storage session has completed, without locking funds in the contract during the Filecoin proof process.

Store data

Once a valid deposit is available, the app/user can push data to the provider using the store endpoint. This simply takes a File (or FormData) object, and send the bytes to the provider for preparation and Filecoin storage. In NodeJS, callers can also provide a Readable stream object that contains the FormData bytes (see tests for examples).

const blob = new Blob(["Hello, world!"], { type: "text/plain" });
const file = new File([blob], "welcome.txt", {
  type: "text/plain",
  lastModified: new Date().getTime()
});

// The store API also takes optional configuration parameters
const { id, cid } = await storage.store(file);

The response from the store method includes the storage request id, and the data content id (or cid).The app should keep track of at least one of these values for later retrieval. For example, the app might be configured to store a mapping of Account Id to cid, to allow users to retrieve their data at a later date. Additionally, the cid can be used to fetch the data over IPFS via any public IPFS gateway, or IPFS node, including browser-based nodes. This makes it really easy to reference the data in NFT assets or any other on-chain reference to off-chain data!

Check status

The status of the file can be queried using its id. The storage process ranges from "batching" files together, to "preparing" the storage deal, to "auctioning" the set of storage deals, to the actual "deal making" and "success" of the final storage deal on Filecoin. Along the way, clients may query the status in order to provide feedback to users.

const { request, deals } = await storage.status(id);
console.log(request.status_code);
console.log(deals); // Array, empty if no deals on chain yet

It is now safe to release the deposit:

await storage.releaseDeposit();

Other APIs

The @textile/near-storage SDK provides a few other endpoints for developers to use, including a JSON Web Signature (JWS) signing utility that let's you create a (modified) JWS token.

Here's an example using the createToken API from a NodeJS script (assumes you have signed in with near login):

import { keyStores, InMemorySigner } from "near-api-js";
import os from "os";
import path from "path";
import { createToken } from "@textile/near-storage";

const keyStore = new keyStores.UnencryptedFileSystemKeyStore(
  path.join(os.homedir(), ".near-credentials")
);
const accountId = "account.testnet";
const networkId = "testnet";
const aud = "provider.testnet"; // Intended audience
const signer = new InMemorySigner(keyStore);

const token = await createToken(signer, { accountId, networkId, aud });

Read the full API documentation here.

Examples

Metrics

Review all storage requests and Filecoin deals here: https://textileio.retool.com/embedded/public/146224ec-962b-42eb-b567-c89c546845ee