Elaborated on high level vision.

[sixth-data.git] / doc / index.org

diff --git a/doc/index.org b/doc/index.org
index 3c93341..62f2957 100644 (file)
--- a/doc/index.org
+++ b/doc/index.org
@@ -17,6 +17,70 @@
 - [[http://svjatoslav.eu/programs.jsp][other applications hosted at svjatoslav.eu]]
 
 
+* (document settings)                                              :noexport:
+** use dark style for TWBS-HTML exporter
+#+HTML_HEAD: <link href="https://bootswatch.com/darkly/bootstrap.min.css" rel="stylesheet">
+#+HTML_HEAD: <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/1.11.2/jquery.min.js"></script>
+#+HTML_HEAD: <script src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.3.1/js/bootstrap.min.js"></script>"
+#+HTML_HEAD: <style type="text/css">
+#+HTML_HEAD:   footer {background-color: #111 !important;}
+#+HTML_HEAD:   pre {background-color: #111; color: #ccc;}
+#+HTML_HEAD: </style>
+
+* Vision / goal
+Provide versioned, clustered, flexible, object-relational database
+functionality for the [[http://www2.svjatoslav.eu/gitbrowse/sixth/doc/index.html][Sixth computation engine]].
+
++ I hate object-relational impedance mismatch.
+
++ I don't like to convert data between persistent database and runtime
+  objects for every transaction. How about creating united
+  database/computation engine instead to:
+  + Eliminate constant moving and converting of data between 2 systems.
+  + Abstract away difference between RAM VS persistent storage. Let
+    the system decide at runtime which data to keep in what kind of
+    memory.
+
+** Inspiration
++ Relational databases:
+  + Transactional.
+  + Indexable / Quickly searchable.
+
++ Git (version control system)
+  + Versionable
+  + Branchable / mergeable.
+  + Transparent cansistency, checksumming and deduplication.
+  + (Git as a database:
+    https://www.kenneth-truyers.net/2016/10/13/git-nosql-database/ )
+
+** Solution (the big idea)
+I see 4D data structure.
+
+[[file:data model.png]]
+
+Dimensions:
++ List of all the objecs in the system (rows).
++ List of all declared unique object fields (columns).
++ List of all historical transactions/commits/versions (think of
+  sheets of paper).
++ List of all concurrently running branches/threads. Branches can
+  appear and merge over time as needed.
++ (Every cell is concrete field value within an object)
+
+Partitioning/clustering:
++ Why not to partition/(load balance) as required across networked
+  physical computers along arbitrary dimension(s) declared above ?
+
+Indexing (for fast searching):
++ Why not to index along arbitrary dimensions (as required) ?
+
+Further optimizations:
++ In current early stage, trying to focus on minimum possible set of
+  features that would provide maximum possible set of power/benefit :)
++ Once featres are locked. Anything can be optimised. Optimization for
+  size (deduplication) can be solved using Git style content
+  addressible storage mechanism.
+
 * Current status
 - Implemented very simple persistent key-value map.