4 <title>Sixth - system for data storage, computation, exploration and interaction</title>
5 <!-- 2017-05-25 Thu 22:58 -->
7 <meta name="viewport" content="width=device-width, initial-scale=1">
8 <meta name="generator" content="Org-mode">
9 <meta name="author" content="Svjatoslav Agejenko">
10 <link href="https://bootswatch.com/darkly/bootstrap.min.css" rel="stylesheet">
11 <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/1.11.2/jquery.min.js"></script>
12 <script src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.3.1/js/bootstrap.min.js"></script>"
13 <style type="text/css">
14 footer {background-color: #111 !important;}
15 pre {background-color: #111; color: #ccc;}
17 <style type="text/css">
18 /* org mode styles on top of twbs */
35 background-color: #f5f5f5;
62 display: inline-block;
74 .bs-docs-sidebar.affix {
77 @media (min-width: 768px) {
83 /* All levels of nav */
84 .bs-docs-sidebar .nav > li > a {
91 .bs-docs-sidebar .nav > li > a:hover,
92 .bs-docs-sidebar .nav > li > a:focus {
95 text-decoration: none;
96 background-color: transparent;
97 border-left: 1px solid #A1283B;
99 .bs-docs-sidebar .nav > .active > a,
100 .bs-docs-sidebar .nav > .active:hover > a,
101 .bs-docs-sidebar .nav > .active:focus > a {
105 background-color: transparent;
106 border-left: 2px solid #A1283B;
109 /* Nav: second level (shown on .active) */
110 .bs-docs-sidebar .nav .nav {
111 display: none; /* Hide by default, but at >768px, show it */
112 padding-bottom: 10px;
114 .bs-docs-sidebar .nav .nav > li > a {
121 .bs-docs-sidebar .nav .nav > li > a:hover,
122 .bs-docs-sidebar .nav .nav > li > a:focus {
125 .bs-docs-sidebar .nav .nav > .active > a,
126 .bs-docs-sidebar .nav .nav > .active:hover > a,
127 .bs-docs-sidebar .nav .nav > .active:focus > a {
132 /* Nav: third level (shown on .active) */
133 .bs-docs-sidebar .nav .nav .nav {
134 padding-bottom: 10px;
136 .bs-docs-sidebar .nav .nav .nav > li > a {
143 .bs-docs-sidebar .nav .nav .nav > li > a:hover,
144 .bs-docs-sidebar .nav .nav .nav > li > a:focus {
147 .bs-docs-sidebar .nav .nav .nav > .active > a,
148 .bs-docs-sidebar .nav .nav .nav > .active:hover > a,
149 .bs-docs-sidebar .nav .nav .nav > .active:focus > a {
154 /* Show and affix the side nav when space allows it */
155 @media (min-width: 992px) {
156 .bs-docs-sidebar .nav > .active > ul {
159 /* Widen the fixed sidebar */
160 .bs-docs-sidebar.affix,
161 .bs-docs-sidebar.affix-bottom {
164 .bs-docs-sidebar.affix {
165 position: fixed; /* Undo the static from mobile first approach */
168 .bs-docs-sidebar.affix-bottom {
169 position: absolute; /* Undo the static from mobile first approach */
171 .bs-docs-sidebar.affix .bs-docs-sidenav,.bs-docs-sidebar.affix-bottom .bs-docs-sidenav {
176 @media (min-width: 1200px) {
177 /* Widen the fixed sidebar again */
178 .bs-docs-sidebar.affix-bottom,
179 .bs-docs-sidebar.affix {
184 <script type="text/javascript">
188 $('.bs-docs-sidebar li').first().addClass('active');
190 $(document.body).scrollspy({target: '.bs-docs-sidebar'});
192 $('.bs-docs-sidebar').affix();
197 <div id="content" class="container">
198 <div class="row"><div class="col-md-9"><h1 class="title">Sixth - system for data storage, computation, exploration and interaction</h1>
201 <li>This is a subproject of <a href="http://www2.svjatoslav.eu/gitbrowse/sixth/doc/index.html">Sixth</a>
204 <li><a href="http://www2.svjatoslav.eu/gitweb/?p=sixth-data.git;a=snapshot;h=HEAD;sf=tgz">download latest snapshot</a>
207 <li>This program is free software; you can redistribute it and/or modify
208 it under the terms of version 3 of the <a href="https://www.gnu.org/licenses/lgpl.html">GNU Lesser General Public
209 License</a> or later as published by the Free Software Foundation.
214 <li>Svjatoslav Agejenko
216 <li>Homepage: <a href="http://svjatoslav.eu">http://svjatoslav.eu</a>
218 <li>Email: <a href="mailto://svjatoslav@svjatoslav.eu">mailto://svjatoslav@svjatoslav.eu</a>
223 <li><a href="http://svjatoslav.eu/programs.jsp">other applications hosted at svjatoslav.eu</a>
228 <div id="outline-container-sec-1" class="outline-2">
229 <h2 id="sec-1"><span class="section-number-2">1</span> Vision / goal</h2>
230 <div class="outline-text-2" id="text-1">
232 Provide versioned, clustered, flexible, object-relational database
233 functionality for the <a href="http://www2.svjatoslav.eu/gitbrowse/sixth/doc/index.html">Sixth computation engine</a>.
237 <li>I hate object-relational impedance mismatch.
240 <li>I don't like to convert data between persistent database and runtime
241 objects for every transaction. How about creating united
242 database/computation engine instead to:
244 <li>Eliminate constant moving and converting of data between 2 systems.
246 <li>Abstract away difference between RAM VS persistent storage. Let
247 the system decide at runtime which data to keep in what kind of
255 <div id="outline-container-sec-1-1" class="outline-3">
256 <h3 id="sec-1-1"><span class="section-number-3">1.1</span> Inspiration</h3>
257 <div class="outline-text-3" id="text-1-1">
259 <li>Relational databases:
263 <li>Indexable / Quickly searchable.
268 <li>Git (version control system)
272 <li>Branchable / mergeable.
274 <li>Transparent cansistency, checksumming and deduplication.
276 <li>(Git as a database:
277 <a href="https://www.kenneth-truyers.net/2016/10/13/git-nosql-database/">https://www.kenneth-truyers.net/2016/10/13/git-nosql-database/</a> )
285 <div id="outline-container-sec-1-2" class="outline-3">
286 <h3 id="sec-1-2"><span class="section-number-3">1.2</span> Solution (the big idea)</h3>
287 <div class="outline-text-3" id="text-1-2">
289 I see 4D data structure.
294 <p><img src="data model.png" class="img-responsive" alt="data model.png">
302 <li>List of all the objecs in the system (rows).
304 <li>List of all declared unique object fields (columns).
306 <li>List of all historical transactions/commits/versions (think of
309 <li>List of all concurrently running branches/threads. Branches can
310 appear and merge over time as needed.
312 <li>(Every cell is concrete field value within an object)
317 Partitioning/clustering:
320 <li>Why not to partition/(load balance) as required across networked
321 physical computers along arbitrary dimension(s) declared above ?
326 Indexing (for fast searching):
329 <li>Why not to index along arbitrary dimensions (as required) ?
334 Further optimizations:
337 <li>In current early stage, trying to focus on minimum possible set of
338 features that would provide maximum possible set of power/benefit :)
340 <li>Once featres are locked. Anything can be optimised. Optimization for
341 size (deduplication) can be solved using Git style content
342 addressible storage mechanism.
349 <div id="outline-container-sec-2" class="outline-2">
350 <h2 id="sec-2"><span class="section-number-2">2</span> Current status</h2>
351 <div class="outline-text-2" id="text-2">
353 <li>Implemented very simple persistent key-value map.
358 Long term goal is to implement more advanced features on top of this.
362 </div><div class="col-md-3"><nav id="table-of-contents">
363 <div id="text-table-of-contents" class="bs-docs-sidebar">
365 <li><a href="#sec-1">1. Vision / goal</a>
367 <li><a href="#sec-1-1">1.1. Inspiration</a></li>
368 <li><a href="#sec-1-2">1.2. Solution (the big idea)</a></li>
371 <li><a href="#sec-2">2. Current status</a></li>
376 <footer id="postamble" class="">
377 <div><p class="author">Author: Svjatoslav Agejenko</p>
378 <p class="date">Created: 2017-05-25 Thu 22:58</p>
379 <p class="creator"><a href="http://www.gnu.org/software/emacs/">Emacs</a> 25.1.1 (<a href="http://orgmode.org">Org-mode</a> 8.2.10)</p>