Cohesive User Experience

Cardstack provides an out-of-the-box user experience, which allows users and developers alike to construct sophisticated applications and workflows, using common building blocks that are integrated into the platform.

Built-in tools that come with every installation of Cardstack:

• Card Editor : This tool allows users to perform data entries on any card, using either built-in user interface components or a custom UI provided by developers.
• Card Schema Builder : This tool allows users and developers to create new data models for brand-new cards or adjust data models derived from preexisting cards, by using a drag-and-drop interface to add fields to new or adopted cards.
• Card Themer : This tool gives developers the freedom to overwrite the look and feel of any card in the system by providing a new style sheet in the CSS language, using an in-browser code editing environment.

Feature extensions via the Card SDK:

• Developers can extend the Cardstack system by integrating other systems and providing unique user experiences. They can create entirely new types of cards, using the Card SDK—which allows for arbitrary layouts, integrations, and business logic.

This is an overview of the components and architecture layers of Cardstack:

Decentralized Application Server

The Cardstack Hub is the decentralized application server underlying the Cardstack technology. It provides a set of services that offer developers and users on-ramps to applications running across multiple blockchains and clouds. With features being contributed by third-party developers, the Cardstack platform aims to provide an open-source alternative to centralized SaaS (Software as a Service).

Cardstack’s technology supports both consumer and enterprise deployment. With regards to consumers, Cardstack is deployed as a no-code application builder platform that is hosted for the convenience of user access. For enterprises, Cardstack is a customizable and extensible decentralized application runtime that can be deployed in the enterprise’s preferred cloud. 

Card SDK

Developers using the Card SDK can create entirely new functionalities, which can work side by side with the built-in features that come with the Cardstack platform. Applications that are created with the Card SDK retain the ability to be assembled through the Card Schema Builder as well as configured and combined by users through familiar point-and-click and drag-and-drop tools.

Customizing templates

The primary components of an application developed through the Card SDK are the user-facing templates.

• Standard layout (no code) : The simplest and quickest way to create custom cards is to use the no-code Cardstack Builder for the basic structure of the card’s data model.
• Themed layout (low code) : Once the basic schema has been created, developers can choose to customize only the look and feel of the standard layout templates, providing CSS that changes the theme of the card without redefining its interaction characteristics and business logic.
• Custom layout (full code) : Developers are free to define the user interactivity entirely, providing the customization equivalent of a bespoke Web application. They can use the Card SDK locally on their machine and leverage the CLI infrastructure to develop JavaScript-based apps, which can be deployed back to the cloud when they are complete. The components of such a fully customized app include:

UI components

• HTML templates (Glimmer component syntax) : Cardstack uses the Glimmer component syntax, also known as angle bracket invocation, as the basic templating mechanism for defining UI logic. While we share a template rendering runtime with the Ember.js project, we have our own routing logic, so that developers only need to define their components, without having to construct the state management system for their JavaScript app.
• CSS (Cascading Style Sheets) : Developers who use CSS frameworks like Sass will need to include those dependencies in their apps, ensuring that their higher-level style sheets can be compiled down when the application is built and deployed by the CLI.
• Client-side JavaScript (Ember Octane edition) : Developers can go beyond basic templating in the Glimmer component syntax—developing an entire front-end JavaScript application, written in the convention of the Ember Octane edition, and exporting the resulting app as a Cardstack card, which is fully compatible with the rest of the prebuilt user environment. Please see the Ember.js guides for documentation on how to use the latest features in Ember’s Octane edition to create high-performance and sophisticated client-side experiences.

Data services

In order to extend a card by integrating it with other back-end services, developers can add server-side features that will be running as Node.js plugins. These plugins can tap into new data sources, act as middleware for custom integrations, or define new computations that are performed when data changes or events occur.

Computed fields

Cardstack implements a functional reactive data processing architecture that will update computed fields—which could be aggregates, formulas, or custom data lookup functions—whenever the underlying dependent fields are changed.

• Cross data source : Computations can depend on data fields from two or more distinct data sources; for example, correlating a user record from a single sign-on system with a record coming from a customer survey system. Data coming from a blockchain / DLT can also be incorporated in these functions. This allows for data summarization and scoring across all data sources that are integrated into the Cardstack Hub.
• User-provided expressions : For basic calculations, users can apply a simple JavaScript-based expression language, in order to define arithmetic calculations and other statistical aggregations without requiring a programmer to develop custom software. These expressions can be entered through the user interface.

Dependencies

Cardstack is built on top of the JavaScript ecosystem; we use the key features and libraries from the npm registry to construct the overall platform. Developers who write custom software on top of Cardstack can apply most npm packages to their work, and our build system will incorporate those dependencies into the final runtime without any modifications.

• Integration : Cardstack uses middleware packages that are available in the npm registry to connect to blockchain, cloud, and enterprise data sources. While we prefer Cardstack’s core packages to be pure JavaScript implementations, we do support npm packages with native dependencies when they are needed.

Schema Definition

Cardstack is a schema-driven application platform, where all data models are defined in terms of either core fields or compound fields. They are composed by users or developers in order to create the domain model with which the user interacts.

Fields

The bottom layer of the Cardstack schema language consists of fields. These fields represent atomic units of information, which support data entry, data persistence, data query, and data display.

• Core fields : Like any database system, Cardstack supports all common core types—such as text, numbers, dates, relationships, and enumerations.
• Compound fields : In addition, we can use these core field types to create compound field types that are closer to the users’ expectations—such as an address field that combines multiple text, number, and enumeration fields.

Cards

A card is created by assembling multiple fields. The card contains the nested data structure of its fields and can move as one unit. For example, a location could be a card that includes the name of the business, the address, and the phone number.

• Adopt : A card can be based on and extend another card’s schema through a technique called “adopt”. This means that the card borrows all the fields from the card it adopts, but adds additional fields that are needed for the particular use case. This card retains the same compatibility as the original card, since they share the same fields; therefore, they are both compatible with other systems that require these exact fields.
• Accept : A field can be configured to accept data of a certain type; this can be a core type like a number, but it could also be a compound type like an article. Any card that is accepted by a field can provide predictable data—such as a thumbnail for an article, which allows the layout engine to display a grid of articles with thumbnails. If an article card is adopted to create a new type of card, e.g. a photo essay, the shared fields are still available to be accepted by cards that expect the article type.

Collections

Fields can be configured to accept more than one item of the same type. For example, a blog is simply a card that accepts one or more articles, which turns it into a collection. A collection provides the capability to set filter criteria, enabling users to decide which subset of cards will be displayed in each field.

• Relationships : Collections define how cards can relate to other cards. A category card is a type of query-based relationship that defines a collection of articles in that particular category (by way of a query on the underlying hub for articles that contain a reference to that category). Relationships can also cross boundaries and servers; they can depend on a set of criteria that is defined and available in another server.
• Flex fields : Fields can refer to cards that exist in a remote location. The flex field feature provided by Cardstack enables a local field to keep a copy or a snapshot of the remote data, which allows users to retrieve and search for that remote data, even if the remote resource is not online. When the resource comes back online, the local copy can be updated. Flex fields are provided at the framework level; users and developers can define how their cards take advantage of flex fields, either by configuring them in the UI or by providing logic that runs in the codebase.

Cardstack Hub Runtime

The Cardstack Hub is a Node.js-based application runtime, which allows Cardstack-based applications to boot up and serve users, either connecting locally on the same machine or remotely over the cloud. This multi-layer, decentralized application server provides all the requisite features for interacting with multiple cloud and blockchain protocols through a cohesive user experience. The services provided by the Cardstack Hub include:

Reactive data grid

To ensure efficient and performant data access in the user experience, the Cardstack Hub creates and continuously updates an operational data grid, which acts as a cache of information that exists in the connected data sources. Yet, this operational data grid provides features beyond a simple data cache, including:

• Functional reactive processing (FRP) : The Cardstack Hub updates data in the operational data grid using a functional reactive approach. Every time data is updated in the data source, the hub reacts to that change, using the query system to extract the data and writing it to the next dependent field within the schema system. This is how data propagates through the data grid, ensuring that access to any card containing external data does not require expensive runtime joins; instead, it is a simple retrieval of previously calculated data.
• Data caching : Data that is processed in the FRP pipeline is cached for speed of data reads; but it is invalidated when conditions demand that those caches are evicted, so as to ensure that the data is consistent and correct at all times. This data caching scheme follows the relationships defined in the schema and accounts for all computations that are dependent on the raw data in the grid.
• Indexing : External data sources are continuously indexed in the hub; when changes are detected in the data source, the raw data is pulled into the hub through a streaming process. If the data source is temporarily unavailable, the indexing resumes from the last completed position.
• Searching : Some data sources are not appropriate for full indexing—such as a large historical-transaction database containing every transaction ever made or a blockchain containing every block ever produced. For these large data sources, the Cardstack Hub uses a searcher approach. This means that the hub requests only the records and fields that are necessary, then caches only the fetched records. For cached records populated through searchers, a time-based expiration policy can be applied; alternatively, a developer can write custom rules to invalidate the cache based on user activity or system signals.
• Cross-source computed properties : Computed properties are defined in JavaScript or via a user-accessible expression language. They are calculated, cached, and invalidated in the same FRP process, according to the same relationship between the cards and the external data sources.
• Querying : Once the external data has been incorporated into the operational data grid, additional indexes are created on a streaming basis, ensuring that the process of querying and searching for the right data set is efficient. Cardstack supports Postgres as the query engine. For advanced search applications, Elasticsearch can also be incorporated as a full-text search engine.
• Versioning : The Cardstack data schema supports querying and indexing across many versions of the same document, enabling users to find the right version of the document and revert to an older version if necessary. External snapshots of remote data can also use this versioning mechanism, so as to allow for the coexistence of a high-performance but possibly stale local copy with an up-to-date but high-latency remote copy.

Secure API generation

Based on the definition of a card’s schema, the Cardstack Hub creates a secure API, which is compliant with the JSON:API standard, for every resource and/or card that is defined in the schema.

• Job queuing : The Cardstack Hub is an asynchronous application server built on the event sourcing model that implements a job queuing system, so as to ensure that both real-time and batched tasks are performed in the right order, with the right service level.
• Transaction writing / signing : When the hub receives a request to create or modify a resource, it identifies the type of back-end this resource will modify. It formulates a transaction to the integrated system and performs the writing of the transaction. Then, it updates the rest of the data querying system with the new state of the external system. For integrations of blockchain transactions, the Cardstack Hub also manages the cryptographical signing of the message during this step.
• Access control : Cardstack uses a grant-based access control system, which gives individual users or groups of users access to all the data or a subset of the data in the operational data grid through a set of rules. The data to determine which user is in which group uses the operational data grid’s FRP processing mechanism. Therefore, developers can learn the mechanism and use it to perform data processing as well as data access control.

Integrated build system

The Cardstack Hub contains a fully integrated build system, which creates new application bundles for the front-end experience. This happens when new cards are encountered for the first time or when users make modifications to a card’s schema that introduce new dependencies, which must be pulled down and built into this new expanded user experience. Cardstack uses the Embroider build system as the runtime and development-time build engine, while webpack is used as the front-end code bundling and delivery mechanism. Custom packages or upstream dependencies required by npm can also be included and packaged using this mechanism.

Cardstack Hub Plugins

The Cardstack system can be extended via hub plugins, which provide connectivity to existing and future protocols running on blockchains or in the cloud. These plugins are written as npm modules and integrated with the Cardstack Hub through the Koa framework, running in the Node.js environment.

Storage and persistence

One of the most common plugins to add to Cardstack is one that supports new types of storage and persistence mechanisms.

• Git : By default, Cardstack stores all user data in a git repository, which provides a versioning file system for capturing the card data model in a serialized JSON document.
• GitHub : In order to make it easier for developers to build on the Cardstack platform, Cardstack is integrated with GitHub as a secondary data source, which allows all user contributions to be stored remotely in a private or public GitHub repository, using a remote git protocol. Developers can make changes to the GitHub repo directly, which will be reflected in the hub—whether it is running locally on the developer’s machine or in a hosted cloud environment.
• Amazon S3 : For larger binary data, Cardstack supports a BLOB (Binary Large Object) API, with Amazon S3 being the default cloud storage service. Yet, the adapter used in the Cardstack Hub supports many other cloud storage services from other cloud vendors as well.
• IPFS : For decentralized object storage, Cardstack has been configured to work with IPFS through the same plugin architecture. This ensures that at least one copy of the IPFS resource is available on the network in the process known as “pinning”. The Cardstack Hub can also provide a cloud-based endpoint behind that IPFS service.

Data source translation

External data sources come in many shapes and formats. All data sources need to be converted to JSON, since that is the format supported by our operational data grid. In some instances, additional decoding of those JSON structures needs to be done, so as to make accessing and querying easier. 

• Smart contract : Many smart contracts have a data structure that reflects the state of the transaction; but these transactions are written in a language that is not clear to users. For these smart contracts, we recommend creating cards with fields that map to the users’ expectations, then write a data source codec to map the output of the smart contract to this user-facing data model.
• File format : Some data sources require access to files that contain metadata within their binary stream; for example, metadata embedded within video, audio, and photos that is not easily accessible without a specialized application. For these file formats, we can run the binary file through a metadata extractor, then map the extracted metadata to fields defined in cards that are easily searched through the operational data grid.
• XML / CSV : When connecting to enterprise data sources, the integration provided by these legacy systems often consists of XML documents or CSV export files. In this case, the data source codec can be invoked in a batch process, so that each roll of the incoming XML or CSV can be converted to a corresponding card, as defined by the schema. 

Automation / orchestration

The Cardstack Hub supports the injection of automated procedures, which react to data changes or other user-initiated actions and perform functions that trigger other actions in other parts of the system.

• System-scoped : Some of these automation rules can be installed to run system-wide—e.g. an automation rule to update caches in a content delivery network (CDN) or to send emails when there is a change in the user state.
• User-scoped : Users can also install specialized automation cards into their workspaces, which get triggered when certain conditions are met; these cards trigger actions for the respective user only. This allows users to configure automation rules and let them run on a server, even if they are not logged into the system. Automation cards can be deployed to groups of users through the grant model as well, which enables administrator control over these automation rules, without making individual changes in each user's account.

Security Model

Cardstack implements a decentralized approach to protecting user information and defining access rights. Instead of building one monolithic service that runs in one cloud environment, the ideal configuration of the Cardstack network involves many instances of Cardstack, each running its own hub with its own operational data grid and user database. Through the coordination of federation protocols—either cloud-based ones like OAuth or blockchain-based ones like synchronized states—these individual hubs can provide their users with a well-defined security boundary, while interacting with a wider network of cooperating nodes.

Realms

The core boundary of Cardstack’s security context is a realm. In the simplest sense, a realm is a hub that is running for one user, and one user only. Within that realm / hub, that user is the administrator who has full access to all the data and integration connected with that hub. Most hubs are multi-tenant hubs, where the same hub infrastructure is shared across many users. In those cases, the realms ensure that each user has his own private repository (in a git sense), while the users may share an index in Postgres for operational efficiency.

Credentials

Users need to have certain credentials to log into a realm. Depending on the configuration of a particular hub, different protocols can be supported for login. 

• OAuth : By default, Cardstack supports the popular OAuth protocol as a primary way to authenticate users.
• GitHub : Developers can use their existing GitHub account to create a Cardstack user account in any hub. As a bonus for logging into Cardstack with their GitHub account, they can allow Cardstack to write their personal data to their private GitHub repository under their GitHub account.
• Auth0 : Cardstack also supports Auth0, a popular single sign-on service for authenticating users of commercial platforms or enterprise services. 

Keys

For interacting with a blockchain / DLT or other cryptographic protocols, Cardstack supports two methods of key signing. 

• Client : We support client-side wallets like MetaMask for signing transactions.
• Custodial : We support custodial wallets that are hosted in a separate blockchain node for signing transactions that belong to an organization. Hardware security module (HSM) support can be integrated via the plugin framework as need arises. 

Membership

Realms can issue portable memberships, so as to honor memberships from other realms without asking users to log in twice. This allows for the creation of ad-hoc realms, such as realms containing message threads between two or more parties that are simultaneously hosted by two different hubs.

Wallet

In order to help users organize all their credentials, memberships, and keys, each hub can provide a wallet service as well as an interface to sort and access each of these wallet items, using the same Card UI paradigm.

Users

Each hub / realm has its own concept of users and therefore hosts its own user database. To protect user privacy, these users’ accounts can be randomly generated and do not need to contain human-readable information. In order to associate a user account with a real person, the person can choose to associate one or more user IDs with a membership issued by a realm that contains remotely accessible personal identification information. The KYC system is independent of the user system. 

Groups

Groups are created in each hub to control access to data of the schema hosted by that hub. Some groups are defined globally—e.g. moderators, editors, or administrators—and published to the common catalog. This ensures that card authors can depend on the existence of those common groups, as they define the access control scheme for their cards. If a card requires a group that is not defined within a realm, the user can map that group to an existing group or define a new list of users who belong to that newly syndicated group. 

Protocol Adapters

Cardstack has defined a specialized set of adapters to integrate with certain similar protocols. These adapters include:

Persistence

Cardstack follows the basic operating principle of a object storage protocol as a way to capture user data and provide a history of amendments. 

• Git
• File System
• Postgres

State synchronization

To synchronize states across multiple hubs, Cardstack uses both a centralized git protocol through GitHub and a decentralized git protocol through an Ethereum-anchored approach called Githereum. This ensures that two different realms have access to the same state, just as if they were sharing a local git repository. 

• GitHub
• Githereum

Authentication

Developers can extend this beyond OAuth, by following the contour of the authentication plugin system to adapt to other authentication protocols. Cardstack is part of the Decentralized Identity Foundation (DIF) and plans to support additional decentralized authentication protocols in the near future. 

• OAuth

Metering and billing

As a way to ensure that users can be billed for the services they use, Cardstack supports the inclusion of metering and billing into the runtime through a logging system. This way, usage can be captured in detailed records and aggregated for billing at a later date.

• CARD Protocol
  
• Tally
  

Integration Plugins

Beyond these common protocol plugins, any blockchain or cloud system can be integrated into Cardstack. These are some examples of integrations that have been done using the Cardstack framework’s indexer, searcher, writer, and middleware plugin mechanisms. 

Blockchain / DLT

The Cardstack Hub supports integrations with any blockchain or DLT system, as long as it supports a standard APl—e.g. RPC or REST API—that allows for the ingestion of transactional histories and the sending of new transactions to the network. 

• Full indexing : The most common type of integration uses Cardstack’s indexing architecture to build and refresh a replica of the blockchain database in the operational data grid. This is recommended for systems in which the DLT represents the data of a business network, where each node needs to have quick access to any transaction that happened in the past. 
• Selective caching : If not all transactions on the blockchain / DLT are relevant, the searcher architecture can be selected instead of an indexer; it will only fetch the records that are relevant to a particular user query. These records are cached based on the policy of the underlying DLT, so that future requests for the same information will be served out of the cache, not directly out of the blockchain.

Smart contract

For smart-contract-enabled DLTs, additional rules have to be written to decode the data structure that is encoded in the smart contract, based on the capability of the contract implementation. 

• ERC20 : For smart contracts that abide by standards like ERC20, one adapter—which knows how to decode the data structure of ERC20 contracts—is sufficient. It can be used for any ERC20-compliant contract on both mainnet and various testnets. 
• Arbitrary ABI : For all other smart contracts, the Cardstack indexer will introspect the Application Binary Interface and create a corresponding data structure in the operational data grid without any additional development. Developers can choose to write additional decoders to convert the introspected data structure into a more convenient and searchable schema with additional code programming. 

Enterprise systems

Cardstack can connect to enterprise systems that sit in the cloud or behind firewalls in a similar way.

• Custom (REST API) : If a legacy system already exposes a REST API, the plugin focuses on remapping CRUD to the corresponding REST endpoints.
• Custom (SOAP API) : For enterprise systems that only support Web surface interfaces like the SOAP protocol, custom development can be done to map Cardstack’s writer and searcher interface to the corresponding method calls in the underlying system.

SaaS APIs

Plugins can be written to expose all the features of a cloud-based SaaS service as one set of capabilities.

• Salesforce.com : We are currently developing a generic Salesforce.com plugin that can handle authentication, data structure mapping, as well as user and group configuration mapping within one module.

Batch data sets

For systems that only support batch imports and exports through batch data feeds, Cardstack supports the use of SFTP as a way to pull down and push up data sets that are injected into or transmitted to enterprise systems.

• XML : The Node.js modules support various levels of XML parsing, which can then map to data manipulation calls in the operational data grid. For most industry-specific XML formats, we recommend the creation of custom XML adapters that map the data to JSON, then use the JSON data format as a way to integrate with the rest of the Cardstack system.
• CSV : Comma-Separated Values are commonly used as data exchange format. Yet, due to the flexible nature of CSV, special care is needed to ensure that the type conversion—from CSV to the internal formats of fields within cards—is carefully mapped, with validation rules being implemented during the data import. The version management capability provided by Cardstack enables users to import CSV in a batch and submit that batch into the repository with one push, which allows rollbacks in case of validation errors. 

Module Registry

The Cardstack module registry is based on the npm registry architecture, which is commonly used in the JavaScript ecosystem. Everything in the Cardstack system is an npm package, including plugins, schema definitions, as well as the modules that comprise the Cardstack Hub and its sub-components. 
‍
Cardstack plans to introduce its own shared catalog of features, called the Card Catalog. This catalog will initially be deployed as a GitHub repository, containing cards that are submitted by users and developers, which can then be installed using an npm command line utility. This repository can also be migrated to Githereum. That way, the process of determining whether a submission makes it to the master branch and becomes available to the broad public can be made by the logic of the smart contract, based on community signals and governance decisions.

Starter set

Every Cardstack installation comes with a starter set of cards, including core fields, base cards, forms, and collections. Additional base cards are currently under design and development; they cover assets, threads, actions, layouts, and rules. These cards are immediately usable by users; but they can also serve as templates for further customizations and extensions by users and developers alike.

Premium decks

The Card Catalog supports the registration of premium decks, which are a set of cards that—when used together—address many concerns of a vertical market or departmental function. Currently, we are exploring the creation of decks that cover the areas of decentralized finance (DeFi), customer relationship management (CRM), and Human Resources (HR).

Deployment Options

Cardstack is designed to be deployed in a range of topologies—from typical cloud deployments as SaaS apps to a decentralized peer-to-peer model, where all users run their own hubs. The Cardstack architecture supports all the topologies named below, without changing the front-end or back-end programming approach.

Standalone cloud (SaaS)

The most common deployment model for Cardstack allows users to choose a cloud provider to run Cardstack as a multi-tenant SaaS offering. This enables users to log into a normal Web app, while having all their data hosted and backed up in a convenient way. Cardstack’s own default hosting service, called Card.Space, is deployed in this manner. 

• Amazon Web Services (AWS)
• Azure
• GCP

Companion to DLT nodes (federated)

Cardstack can also be deployed in a federated manner, where multiple nodes run in a network, providing several user on-ramps to the same blockchain / DLT network. In this configuration, the blockchain provides data synchronization, membership access control, as well as cross-node messaging and a public key infrastructure (PKI). The Cardstack architecture currently supports the following blockchains / DLT networks:

• Ethereum Virtual Machine
• Corda
• Hyperledger (legacy)

Local / P2P (developer mode)

For most developers, Cardstack will likely run on the local developer machine, which runs all the open-source code in a transparent way. Users who value ultimate control and privacy can use a similar approach to run their own version of Cardstack on their own local machine. In that configuration, any data that is captured by the Cardstack Hub and stored in the reactive data grid is only visible to the user who has physical access to that machine. In order to interact with other realms that are hosted in other ways, that user has to opt into cross-honored memberships, which use a cryptographically based protocol to exchange messages between hubs, so that information can be exchanged without compromising user privacy.