X-Git-Url: http://www2.svjatoslav.eu/gitweb/?a=blobdiff_plain;f=doc%2Findex.html;h=28f924f76fc0583f20e9a4a859579d8fe4f49dcf;hb=bb7b2daf4049f53eefbde9912daca7f31a3b6717;hp=8793e4565e2e7e39d3822467b63f6e6002838b0f;hpb=45c3bd94e38fc74268429d4441e730aec28492bf;p=sixth-data.git diff --git a/doc/index.html b/doc/index.html index 8793e45..28f924f 100644 --- a/doc/index.html +++ b/doc/index.html @@ -1,350 +1,401 @@ - - - + + - - - - Sixth - system for data storage, computation, exploration and interaction - - - - - +Sixth - system for data storage, computation, exploration and interaction + + + + + + + +" + + + -
-

Sixth - system for data storage, computation, exploration and interaction

-
-

Table of Contents

- -
-
- - - -
-

1 Current status

-
-
    -
  • Implemented very simple persistent key-value map.
  • -
- -

- Long term goal is to implement more advanced features on top of this. -

-
-
+
+

Sixth - system for data storage, computation, exploration and interaction

+
+ + + +
+

1 Vision / goal

+
+

+Provide versioned, clustered, flexible, object-relational database +functionality for the Sixth computation engine. +

+ +
    +
  • I hate object-relational impedance mismatch. +
  • + +
  • I don't like to convert data between persistent database and runtime +objects for every transaction. How about creating united +database/computation engine instead to: +
      +
    • Eliminate constant moving and converting of data between 2 systems. +
    • +
    • Abstract away difference between RAM VS persistent storage. Let +the system decide at runtime which data to keep in what kind of +memory. +
    • +
    +
  • +
-
-

Author: Svjatoslav Agejenko

-

Created: 2016-08-03 Wed 23:45

-

Validate

+ +
+

1.1 Inspiration

+
+
    +
  • Relational databases: +
      +
    • Transactional. +
    • +
    • Indexable / Quickly searchable. +
    • +
    +
  • + +
  • Git (version control system) + +
  • +
+
+
+ +
+

1.2 Solution (the big idea)

+
+

+I see 4D data structure. +

+ + +
+

data model.png +

+
+ +

+Dimensions: +

+
    +
  • List of all the objecs in the system (rows). +
  • +
  • List of all declared unique object fields (columns). +
  • +
  • List of all historical transactions/commits/versions (think of +sheets of paper). +
  • +
  • List of all concurrently running branches/threads. Branches can +appear and merge over time as needed. +
  • +
  • (Every cell is concrete field value within an object) +
  • +
+ +

+Partitioning/clustering: +

+
    +
  • Why not to partition/(load balance) as required across networked +physical computers along arbitrary dimension(s) declared above ? +
  • +
+ +

+Indexing (for fast searching): +

+
    +
  • Why not to index along arbitrary dimensions (as required) ? +
  • +
+ +

+Further optimizations: +

+
    +
  • In current early stage, trying to focus on minimum possible set of +features that would provide maximum possible set of power/benefit :) +
  • +
  • Once featres are locked. Anything can be optimised. Optimization for +size (deduplication) can be solved using Git style content +addressible storage mechanism. +
  • +
+
+
+
+
+

2 Current status

+
+
    +
  • Implemented very simple persistent key-value map. +
  • +
+ +

+Long term goal is to implement more advanced features on top of this. +

+
+
+ +
+

3 TODO

+
+
+

3.1 check out Magma

+ +
+
+
+