1 #+TITLE: Sixth - system for data storage, computation, exploration and interaction
4 - This is a subproject of [[http://www2.svjatoslav.eu/gitbrowse/sixth/doc/index.html][Sixth]]
6 - [[http://www2.svjatoslav.eu/gitweb/?p=sixth-data.git;a=snapshot;h=HEAD;sf=tgz][download latest snapshot]]
8 - This program is free software; you can redistribute it and/or modify
9 it under the terms of version 3 of the [[https://www.gnu.org/licenses/lgpl.html][GNU Lesser General Public
10 License]] or later as published by the Free Software Foundation.
14 - Homepage: http://svjatoslav.eu
15 - Email: mailto://svjatoslav@svjatoslav.eu
17 - [[http://svjatoslav.eu/programs.jsp][other applications hosted at svjatoslav.eu]]
20 * (document settings) :noexport:
21 ** use dark style for TWBS-HTML exporter
22 #+HTML_HEAD: <link href="https://bootswatch.com/darkly/bootstrap.min.css" rel="stylesheet">
23 #+HTML_HEAD: <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/1.11.2/jquery.min.js"></script>
24 #+HTML_HEAD: <script src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.3.1/js/bootstrap.min.js"></script>"
25 #+HTML_HEAD: <style type="text/css">
26 #+HTML_HEAD: footer {background-color: #111 !important;}
27 #+HTML_HEAD: pre {background-color: #111; color: #ccc;}
31 Provide versioned, clustered, flexible, object-relational database
32 functionality for the [[http://www2.svjatoslav.eu/gitbrowse/sixth/doc/index.html][Sixth computation engine]].
34 + I hate object-relational impedance mismatch.
36 + I don't like to convert data between persistent database and runtime
37 objects for every transaction. How about creating united
38 database/computation engine instead to:
39 + Eliminate constant moving and converting of data between 2 systems.
40 + Abstract away difference between RAM VS persistent storage. Let
41 the system decide at runtime which data to keep in what kind of
45 + Relational databases:
47 + Indexable / Quickly searchable.
49 + Git (version control system)
51 + Branchable / mergeable.
52 + Transparent cansistency, checksumming and deduplication.
54 https://www.kenneth-truyers.net/2016/10/13/git-nosql-database/ )
56 ** Solution (the big idea)
57 I see 4D data structure.
59 [[file:data model.png]]
62 + List of all the objecs in the system (rows).
63 + List of all declared unique object fields (columns).
64 + List of all historical transactions/commits/versions (think of
66 + List of all concurrently running branches/threads. Branches can
67 appear and merge over time as needed.
68 + (Every cell is concrete field value within an object)
70 Partitioning/clustering:
71 + Why not to partition/(load balance) as required across networked
72 physical computers along arbitrary dimension(s) declared above ?
74 Indexing (for fast searching):
75 + Why not to index along arbitrary dimensions (as required) ?
77 Further optimizations:
78 + In current early stage, trying to focus on minimum possible set of
79 features that would provide maximum possible set of power/benefit :)
80 + Once featres are locked. Anything can be optimised. Optimization for
81 size (deduplication) can be solved using Git style content
82 addressible storage mechanism.
84 - Implemented very simple persistent key-value map.
86 Long term goal is to implement more advanced features on top of this.
90 + http://wiki.squeak.org/squeak/2665