Data Store API (original) (raw)

The Data Store API allows an application to manage a data store that can be shared with other applications and provides a mechanism to allow multiple applications to concurrently synchronize data from the data store into the application-local cache.

Introduction

The Data Store API allows an application to create and maintain a data store that can be shared with other applications. To make multiple applications keep their local storages up-to-date with the data store, this API supports a mechanism to allow an application to concurrently synchronize data from the data store into the application-local storage which can be indexed, grouped or sorted in whatever format the application needs in order to support its UI.

Motivation

The Contacts Manager API and the Messaging API were originally designed to support richer querying, like filtering, sorting and grouping data. However, they both have the severe shortcoming that the consumers are forced to live with the limitations of what querying capabilities those APIs can have.

That's why we are constantly having to revise these APIs because it turns out that the querying capabilities aren't matching what our applications need, which is not a workable long-term solution. Also, it's not even a workable short-term solution for third-party applications since we cannot revise the APIs to support the capabilities that every third-party application developer needs.

Therefore, we've been figuring out such a generic Data Store API to allow applications to synchronize and save data into their own application-local storages which can be managed in various formats than we could think of and bake into APIs, thus supporting richer querying capabilities that the application really needs.

Note that the Data Store API is still in charge of maintaining a central storage to keep data which can be added, retrieved or deleted by applications through a bunch of methods provided by the Data Store API. This implies the application doesn't need to duplicate the whole data in its own local storage. Instead, it can simply synchronize the data that is actively required to construct the needed indexes for its own querying purpose.

More than IndexedDB API

Similar to the IndexedDB API, the Data Store API provides some database-like methods to add, retrieve and delete data records but it doesn't expose methods for building indexes or searching data. Instead, it provides a synchronizing mechanism for applications to keep their local storages up-to-date and accordingly update the local indexes needed for filtering data.

The other difference from the IndexedDB API is the Data Store API provides a central data storage which can be concurrently accessed and modified by multiple applications. Also, it provides a permission model to allow different applications to have different types of priviledges to change the data store or listen to the change of the data store.

Examples

If an application has the privilege to modify the content of the data store, it can use a bunch of basic methods defined in theDataStore to manage the data records in the data store, which is shown as the following example.

      // Retrieve a list of data stores named as 'fb-contacts'.
      navigator.getDataStores('fb-contacts').then(function(stores) {
        if (!stores.length) {
          return;
        }

        // Check if the application is allowed to modify the data store.
        if (stores[0].readOnly) {
          return;
        }

        // Retrieve an object.
        stores[0].get(42).then(function(obj) {
          // Update the object
          obj.name = 'foo';
          stores[0].update(42, obj).then(function(id) {
            // The object has been updated.
          }, function(error) {
            // The object fails to be updated.
          });
        });
  
        // Delete an object.
        stores[0].remove(23).then(function(success) {
          if (success) {
            // The object has been deleted.
          } else {
            // The object fails to be deleted.
          }
        });
  
        // Add a new object.
        stores[0].insert({ name: "bar" }).then(function(id) {
          // The object has been added.
        }, function(error) {
          // The object fails to be added.
        });
      });

An application can call the sync() method defined in theDataStore to keep its local storage synchronized with the data store, which creates a DataStoreCursor to retrieve the change history starting from a certain revision kept in the application to the current revision of the data store, which is shown as the following example.

      var appLocalRevisionId = "revision_id_kept_by_app";

      // Retrieve a list of data stores named as 'fb-contacts'.
      navagiator.getDataStores('fb-contacts').then(functions(stores) {
        if (!stores.length) {
          return;
        }

        // Check if the application's local storage is out-of-date.
        if (appLocalRevisionId == stores[0].revisionId) {
          dump("The app's local storage is already in sync.\n");
          return;
        }

        var cursor = stores[0].sync(appLocalRevisionId);

        function cursorResolve(task) {
          switch (task.operation) {
            case 'done':
              // All the data are in sync. Update the local revision ID.
              dump("The current revision ID: " + task.revisionId + "\n");
              appLocalRevisionId = task.revisionId;
              return;

            case 'clear':
              // All the data have to be deleted in the local storage.
              break;

            case 'add':
              // A new object has to be added in the local storage.
              dump("Add ID: " + task.id + " data: " + task.data + "\n");
              break;

            case 'update':
              // An object has to be updated in the local storage.
              dump("Update ID: " + task.id + " data: " + task.data + "\n");
              break;

            case 'remove':
              // An object has to be deleted in the local storage.
              dump("Remove ID: " + task.id + " data: " + task.data + "\n");
              break;
          }

          cursor.next().then(cursorResolve);
        }

        // Start to sync.
        cursor.next().then(cursorResolve);
      });

This specification defines conformance criteria that apply to a single product: the user agent that implements the interfaces that it contains.

Implementations that use ECMAScript to implement the APIs defined in this specification MUST implement them in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification [[!WEBIDL]], as this specification uses that specification and terminology.

Terminology

The EventHandler interface represents a callback used for event handlers as defined in [[!HTML5]].

The concepts queue a task and fire an event are defined in [[!HTML5]].

The terms event handler and event handler event types are defined in [[!HTML5]].

The Promise interface, the concepts of a resolver, a resolver's fulfill algorithm and a resolver's reject algorithm are defined in [[DOM4]].

Security and privacy considerations

Application Manifest

For the application that provides the data store, its manifest MUST be claimed to own the data store by datastores-owned, where the JSON object can contain multiple properties representing different names of data stores respectively. Each data store can usereadonly to specify whether the data store can be read by other applications and description to describe the purpose.

As shown as the following example, if a Facebook applicaton wants to provide a read-only fb-contacts data store, its manifest MUST be claimed to own the data store by setting the attributereadonly to true.

      {
        datastores-owned: {
          "fb-contacts": {
            "readonly": true,
            "description": "own the Facebook contacts data store"
          }
        }
      }

For the application that wants to access the data store, its manifest MUST be claimed to access the data store bydatastores-access, where the JSON object can contain multiple properties representing different names of data stores respectively. Each data store can use access to specify the application's accessibility and description to describe the purpose.

As shown as the following example, if the application wants to read or modify (e.g., add, update, remove, clear... etc) the content of the_fb-contacts_ data store, its manifest MUST be claimed to access the fb-contacts data store by setting the attributeaccess to readwrite.

      {
        datastores-access: {
          "fb-contacts": {
            "access": "readwrite",
            "description": "access (read and write) the Facebook contacts data stores"
          }
        }
      }

As shown as the following example, if the application simply wants to read the content of the fb-contacts data store without the need of modifying it, its manifest MUST be claimed to access the_fb-contacts_ data store by setting the attributeaccess to readonly.

      {
        datastores-access: {
          "fb-contacts": {
            "access": "readonly",
            "description": "access (read only) the Facebook contacts data store"
          }
        }
      }

Permission Model

The Data Store API can be exposed to trusted or untrusted contents.

Need to define the permission model of how the privileged and the third-party applications can be allowed to use the Data Store API to access the data store.

Promise getDataStores ()

This method makes a request to retrieve the data stores by thename parameter. It returns a newPromise that will be used to notify the caller about the result of the operation, which is an array of DataStore elements to access the data stores which have the same name equal to the name.

DOMString name

Specifies the name of the data store.

Steps

The getDataStores method when invoked MUST run the following steps:

  1. Let promise be a new Promise object andresolver be its associated resolver.
  2. Return promise to the caller.
  3. Make a request to the system to retrieve the data store(s) with the name equal to the name parameter passed in the request, where the caller application has claimed the data store access by datastores-access in its manifest.
  4. If an error occurs invoke resolver's reject algorithm with error as the value argument.
  5. When the request has been successfully completed:
    1. Let dataStores be the array of the retrievedDataStore elements.
    2. Invoke resolver's fulfill algorithm with dataStores as the value argument.

DataStore Interface

The DataStore interface represents a bunch of properties of the data store and a set of operations that can be used to manage and synchronize the content of the data store.

readonly attribute DOMString name

MUST return the name of the data store. Note that different data stores can share the same name as long as they have the same database schema/format.

readonly attribute DOMString owner

MUST return the owner of the data store, which can be the manifest URL of the owner application.

readonly attribute boolean readOnly

MUST return whether the content of the data store can be changed or not by the caller.

readonly attribute DOMString revisionId

MUST return the current revision of the data store, which can be a UUID string.

Promise get ()

This method makes a request to retrieve the data record(s) by theid parameter. It returns a new Promise that will be used to notify the caller about the result of the operation, which is an arbitrary object to represent the data record if id is a single value, or a set of data records if id is an array of values.

DataStoreKey... id

Identifies the data record(s) that is requested to be retrieved.

Promise update ()

This method makes a request to update the existing data record by the id and the data parameters. It returns a new Promise that will be used to notify the caller about the result of the operation.

DataStoreKey id

Identifies the data record that is requested to be updated.

any data

Specifies the content of the data record to update.

Promise insert ()

This method makes a request to add a new data record by thedata parameter. It returns a new Promise that will be used to notify the caller about the result of the operation, which is an identifier to access the data record that is added.

any data

Specifies the content of the data record to add.

Promise remove ()

This method makes a request to delete the data record(s) by theid parameter. It returns a new Promise that will be used to notify the caller about the result of the operation, which is a boolean value to indicate whether the data record(s) is successfully deleted or not.

DataStoreKey... id

Identifies the data record(s) that is requested to be deleted.

Promise clear ()

This method makes a request to clear all the data records in the data store. It returns a new Promise that will be used to notify the caller about the result of the operation.

Promise getLength ()

This method makes a request to retrieve the total number of data records saved in the data store. It returns a newPromise that will be used to notify the caller about the result of the operation, which is a numeric value to indicate the total number of data records saved in the data store.

DataStoreCursor sync ()

This method makes a request to retrieve the change history between a particular revision and the current revision of the data store by therevisionId parameter. It returns a newDataStoreCursor that will be used to iteratively access a set of DataStoreTask elements.

optional DOMString revisionId

Identifies the revision of the data store that the application currently keeps. If this parameter is absent or cannot be identified, this method will return a DataStoreCursor iterating through all the existing data records currently saved in the data store.

attribute EventHandler onchange

Handles the change event of typeDataStoreChangeEvent, fired when a data record is added, updated or deleted in the data store. Note that if some data change in the data store when the DataStoreCursor is still synchronizing data and the cursor's close method has not yet been called, all the change events will not be dispatched to the application's onchange event handler. Instead, all the changes will be managed by the cursor'snext method as additional operations.

Steps

The get method when invoked MUST run the following steps:

  1. Let promise be a new Promise object andresolver be its associated resolver.
  2. Return promise to the caller.
  3. Make a request to the data store to retrieve the data record(s) with the identifier equal to the id parameter passed in the request.
  4. If an error occurs invoke resolver's reject algorithm with error as the value argument.
  5. When the request has been successfully completed:
    1. Let dataRecord be the retrieved data record(s).
    2. Invoke resolver's fulfill algorithm with dataRecord as the value argument.

The update method when invoked MUST run the following steps:

  1. Let promise be a new Promise object andresolver be its associated resolver.
  2. Return promise to the caller.
  3. If the readOnly attribute of the current data store is_true_ to the caller, invoke resolver's reject algorithm without assigning a value to the value argument.
  4. Make a request to the data store to retrieve the data record with the identifier equal to the id parameter passed in the request.
  5. If an error occurs invoke resolver's reject algorithm with error as the value argument.
  6. When the request has been successfully completed:
    1. Let dataRecord be the retrieved data record.
    2. Replace dataRecord by the data parameter passed in the request.
    3. Make a request to the data store to save dataRecord back to the data store.
    4. If an error occurs invoke resolver's reject algorithm with error as the value argument.
    5. When the request has been successfully completed:
      1. Invoke resolver's fulfill algorithm without assigning a value to the value argument.

The insert method when invoked MUST run the following steps:

  1. Let promise be a new Promise object andresolver be its associated resolver.
  2. Return promise to the caller.
  3. If the readOnly attribute of the current data store is_true_ to the caller, invoke resolver's reject algorithm without assigning a value to the value argument.
  4. Make a request to the data store to add the data parameter passed in the request.
  5. If an error occurs invoke resolver's reject algorithm with error as the value argument.
  6. When the request has been successfully completed:
    1. Let id be the generated identifier of the added data record.
    2. Invoke resolver's fulfill algorithm with id as the value argument.

The remove method when invoked MUST run the following steps:

  1. Let promise be a new Promise object andresolver be its associated resolver.
  2. Return promise to the caller.
  3. If the readOnly attribute of the current data store is_true_ to the caller, invoke resolver's reject algorithm with false as the value argument.
  4. Make a request to the data store to retrieve the data record(s) with the identifier equal to the id parameter passed in the request.
  5. If an error occurs invoke resolver's reject algorithm with false as the value argument.
  6. When the request has been successfully completed:
    1. Make a request to the data store to delete the data record(s) with the identifier equal to the id parameter passed in the request.
    2. If an error occurs invoke resolver's reject algorithm with false as the value argument.
    3. When the request has been successfully completed:
      1. Invoke resolver's fulfill algorithm with true as the value argument.

The clear method when invoked MUST run the following steps:

  1. Let promise be a new Promise object andresolver be its associated resolver.
  2. Return promise to the caller.
  3. If the readOnly attribute of the current data store is_true_ to the caller, invoke resolver's reject algorithm without assigning a value to the value argument.
  4. Make a request to the data store to clear all the data records.
  5. If an error occurs invoke resolver's reject algorithm with error as the value argument.
  6. When the request has been successfully completed:
    1. Invoke resolver's fulfill algorithm without assigning a value to the value argument.

The getLength method when invoked MUST run the following steps:

  1. Let promise be a new Promise object andresolver be its associated resolver.
  2. Return promise to the caller.
  3. Make a request to the data store to get the total number of data records.
  4. If an error occurs invoke resolver's reject algorithm with error as the value argument.
  5. When the request has been successfully completed:
    1. Let length be the total number of data records saved in the data store.
    2. Invoke resolver's fulfill algorithm with length as the value argument.

The sync method when invoked MUST run the following steps:

  1. Make a request to the system to retrieve a cursor that can iterate through the change history of the current data store.
  2. When the request has been successfully completed:
    1. Let dataStoreCursor be a new instance ofDataStoreCursor.
    2. Set the store of dataStoreCursor to the current data store.
    3. Return dataStoreCursor.

Event handlers

The following are the event handlers (and their corresponding event types) that MUST be supported as attributes by the DataStore object.

Event handler Event name Event type Short description
onchange change DataStoreChangeEvent Handles the information of the data record changed in the data store.

DataStoreCursor Interface

The DataStoreCursor interface allows the application to iterate through a list of DataStoreTask elements that represents the change history of the data store.

readonly attribute DataStore store

MUST return the data store that is currently iterated by the cursor.

Promise next ()

This method makes a request to retrieve the information of the next operation that changes a data record in the data store. It returns a new Promise that will be used to notify the caller about the result of the operation, which is a DataStoreTask to represent the information of the change operation.

void close ()

This method makes a request to terminate the cursor iterating through the change history of the data store. Note that this method has to be explicitly called when the cursor completes its tasks. Otherwise, if some data changes in the data store when the cursor is still synchronizing data, all the changes will be managed by the cursor's next method as additional operations, which means when the cursor completes its tasks, the application will be in synchronization with the current revision of the data store.

Steps

The next method when invoked MUST run the following steps:

  1. Let promise be a new Promise object andresolver be its associated resolver.
  2. Return promise to the caller.
  3. Make a request to the system to retrieve the information of the next operation that changes a data record in the data store.
  4. If an error occurs invoke resolver's reject algorithm with error as the value argument.
  5. When the request has been successfully completed:
    1. Let dataStoreTask be a new instance ofDataStoreTask:
      1. Set the revisionId of dataStoreTask to the revision of the data store, which changes a data record.
      2. Set the id of dataStoreTask to the identifier of the changed data record in the data store.
      3. Set the operation of dataStoreTask to the type of operation that changes the data record in the data store.
      4. Set the data of dataStoreTask to the changed data record in the data store.
    2. Invoke resolver's fulfill algorithm with dataStoreTask as the value argument.

The close method when invoked MUST run the following steps:

  1. Make a request to the system to terminate the current cursor iterating through the change history of the data store.

DataStoreTask Dictionary

The DataStoreTask dictionary contains the information related to a data record changed in the data store.

DOMString revisionId

MUST return the identifier of the revision of the data store, which changes a data record.

DataStoreOperation operation

MUST return the type of operation that changes the data record in the data store.

DataStoreKey? id

MUST return the identifier of the changed data record in the data store. MUST return null if the operation is clear or done.

any data

MUST return an arbitrary object to represent the changed data record in the data store. MUST return null if theoperation is clear or done.

DataStoreChangeEvent Interface

The DataStoreChangeEvent interface represents the event related to a date record changed in the data store.

readonly attribute DOMString revisionId

MUST return the identifier of the revision of the data store, which changes a data record.

readonly attribute DataStoreOperation operation

MUST return the type of operation that changes the data record in the data store.

readonly attribute DataStoreKey? id

MUST return the identifier of the changed data record in the data store. MUST return null if the operation is_clear_ or done.

readonly attribute DOMString owner

MUST return the manifest URL of the application which changes the data record in the data store. Note that the owner here is not the owner application of the data store. Instead, it's the application making the change in the data store.

Enumerations

The attribute operation in a DataStoreTask or a DataStoreChangeEvent can have the following values:

add

The data record is added in the data store.

update

The data record is updated in the data store.

remove

The data record is deleted in the data store.

clear

All the data records are deleted in the data store.

done

No additional opreations in the data store.

Acknowledgements

The editors would like to express their gratitude to the Mozilla Firefox OS Team and specially to Jonas Sicking, Mounir Lamouri, Ehsan Akhgari, Thinker Lee and Hsin-Yi Tsai for their technical guidance, as well as to Andrea Marchesini for his implementation work and support. Also, huge thanks to Zoltan Kis (Intel) and Christophe Dumez (Samsung) for their suggestions and contributions to this specification.