X-Git-Url: http://www2.svjatoslav.eu/gitweb/?p=sixth-data.git;a=blobdiff_plain;f=doc%2Findex.html;h=cc7cce511bd10eec26e31452391debda615c27db;hp=29e212a12b7257c69acc942f32670f0e59ad3def;hb=2316187f481ff4854fd93e381e6b1c802cd5bac0;hpb=aa42d5083263f7e12d41912e7094dc35745c489f diff --git a/doc/index.html b/doc/index.html index 29e212a..cc7cce5 100644 --- a/doc/index.html +++ b/doc/index.html @@ -1,184 +1,357 @@ - - - + + - - - Sixth - system for data storage, computation, exploration and interaction - - + + + + + + + +" + - -
-

Sixth - system for data storage, computation, exploration and interaction

-
-

Table of Contents

-
- -
-
-
+
+

Sixth - system for data storage, computation, exploration and interaction

+
+ -
  • other applications hosted at svjatoslav.eu
    • +
  • other applications hosted at svjatoslav.eu +
    • -
    -

    1 Current status

    +
    +

    1 Vision / goal

    +

    +Provide versioned, clustered, flexible, object-relational database +functionality for the Sixth computation engine. +

    + +
      +
    • I hate object-relational impedance mismatch. +
      • + +
    • I don't like to convert data between persistent database and runtime +objects for every transaction. How about creating united +database/computation engine instead to: +
        +
      • Eliminate constant moving and converting of data between 2 systems. +
        • +
      • Abstract away difference between RAM VS persistent storage. Let +the system decide at runtime which data to keep in what kind of +memory. +
        • +
      +
      • +
    +
    + +
    +

    1.1 Inspiration

    +
    +
      +
    • Relational databases: +
        +
      • Transactional. +
        • +
      • Indexable / Quickly searchable. +
        • +
      +
      • + +
    • Git (version control system) + +
      • +
    +
    +
    + +
    +

    1.2 Solution (the big idea)

    +
    +

    +I see 4D data structure. +

    + + +
    +

    data model.png +

    +
    + +

    +Dimensions: +

    +
      +
    • List of all the objecs in the system (rows). +
      • +
    • List of all declared unique object fields (columns). +
      • +
    • List of all historical transactions/commits/versions (think of +sheets of paper). +
      • +
    • List of all concurrently running branches/threads. Branches can +appear and merge over time as needed. +
      • +
    • (Every cell is concrete field value within an object) +
      • +
    + +

    +Partitioning/clustering: +

    +
      +
    • Why not to partition/(load balance) as required across networked +physical computers along arbitrary dimension(s) declared above ? +
      • +
    + +

    +Indexing (for fast searching): +

      -
    • Implemented very simple persistent key-value map.
      • +
    • Why not to index along arbitrary dimensions (as required) ? +
      • +
    + +

    +Further optimizations: +

    +
      +
    • In current early stage, trying to focus on minimum possible set of +features that would provide maximum possible set of power/benefit :) +
      • +
    • Once featres are locked. Anything can be optimised. Optimization for +size (deduplication) can be solved using Git style content +addressible storage mechanism. +
      • +
    +
    +
    +
    + +
    +

    2 Current status

    +
    +
      +
    • Implemented very simple persistent key-value map. +

    @@ -186,11 +359,25 @@ Long term goal is to implement more advanced features on top of this.

    +
    +
    +