1 <?xml version="1.0" encoding="utf-8"?>
2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
3 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
4 <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
6 <!-- 2022-10-30 Sun 12:09 -->
7 <meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
8 <meta name="viewport" content="width=device-width, initial-scale=1" />
9 <title>Sixth Data - Data storage and computing engine</title>
10 <meta name="author" content="Svjatoslav Agejenko" />
11 <meta name="generator" content="Org Mode" />
13 #content { max-width: 60em; margin: auto; }
14 .title { text-align: center;
15 margin-bottom: .2em; }
16 .subtitle { text-align: center;
20 .todo { font-family: monospace; color: red; }
21 .done { font-family: monospace; color: green; }
22 .priority { font-family: monospace; color: orange; }
23 .tag { background-color: #eee; font-family: monospace;
24 padding: 2px; font-size: 80%; font-weight: normal; }
25 .timestamp { color: #bebebe; }
26 .timestamp-kwd { color: #5f9ea0; }
27 .org-right { margin-left: auto; margin-right: 0px; text-align: right; }
28 .org-left { margin-left: 0px; margin-right: auto; text-align: left; }
29 .org-center { margin-left: auto; margin-right: auto; text-align: center; }
30 .underline { text-decoration: underline; }
31 #postamble p, #preamble p { font-size: 90%; margin: .2em; }
32 p.verse { margin-left: 3%; }
34 border: 1px solid #e6e6e6;
36 background-color: #f2f2f2;
38 font-family: monospace;
53 background-color: #f2f2f299;
55 pre.src:hover:before { display: inline; margin-top: 14px;}
56 /* Languages per Org manual */
57 pre.src-asymptote:before { content: 'Asymptote'; }
58 pre.src-awk:before { content: 'Awk'; }
59 pre.src-authinfo::before { content: 'Authinfo'; }
60 pre.src-C:before { content: 'C'; }
61 /* pre.src-C++ doesn't work in CSS */
62 pre.src-clojure:before { content: 'Clojure'; }
63 pre.src-css:before { content: 'CSS'; }
64 pre.src-D:before { content: 'D'; }
65 pre.src-ditaa:before { content: 'ditaa'; }
66 pre.src-dot:before { content: 'Graphviz'; }
67 pre.src-calc:before { content: 'Emacs Calc'; }
68 pre.src-emacs-lisp:before { content: 'Emacs Lisp'; }
69 pre.src-fortran:before { content: 'Fortran'; }
70 pre.src-gnuplot:before { content: 'gnuplot'; }
71 pre.src-haskell:before { content: 'Haskell'; }
72 pre.src-hledger:before { content: 'hledger'; }
73 pre.src-java:before { content: 'Java'; }
74 pre.src-js:before { content: 'Javascript'; }
75 pre.src-latex:before { content: 'LaTeX'; }
76 pre.src-ledger:before { content: 'Ledger'; }
77 pre.src-lisp:before { content: 'Lisp'; }
78 pre.src-lilypond:before { content: 'Lilypond'; }
79 pre.src-lua:before { content: 'Lua'; }
80 pre.src-matlab:before { content: 'MATLAB'; }
81 pre.src-mscgen:before { content: 'Mscgen'; }
82 pre.src-ocaml:before { content: 'Objective Caml'; }
83 pre.src-octave:before { content: 'Octave'; }
84 pre.src-org:before { content: 'Org mode'; }
85 pre.src-oz:before { content: 'OZ'; }
86 pre.src-plantuml:before { content: 'Plantuml'; }
87 pre.src-processing:before { content: 'Processing.js'; }
88 pre.src-python:before { content: 'Python'; }
89 pre.src-R:before { content: 'R'; }
90 pre.src-ruby:before { content: 'Ruby'; }
91 pre.src-sass:before { content: 'Sass'; }
92 pre.src-scheme:before { content: 'Scheme'; }
93 pre.src-screen:before { content: 'Gnu Screen'; }
94 pre.src-sed:before { content: 'Sed'; }
95 pre.src-sh:before { content: 'shell'; }
96 pre.src-sql:before { content: 'SQL'; }
97 pre.src-sqlite:before { content: 'SQLite'; }
98 /* additional languages in org.el's org-babel-load-languages alist */
99 pre.src-forth:before { content: 'Forth'; }
100 pre.src-io:before { content: 'IO'; }
101 pre.src-J:before { content: 'J'; }
102 pre.src-makefile:before { content: 'Makefile'; }
103 pre.src-maxima:before { content: 'Maxima'; }
104 pre.src-perl:before { content: 'Perl'; }
105 pre.src-picolisp:before { content: 'Pico Lisp'; }
106 pre.src-scala:before { content: 'Scala'; }
107 pre.src-shell:before { content: 'Shell Script'; }
108 pre.src-ebnf2ps:before { content: 'ebfn2ps'; }
109 /* additional language identifiers per "defun org-babel-execute"
111 pre.src-cpp:before { content: 'C++'; }
112 pre.src-abc:before { content: 'ABC'; }
113 pre.src-coq:before { content: 'Coq'; }
114 pre.src-groovy:before { content: 'Groovy'; }
115 /* additional language identifiers from org-babel-shell-names in
116 ob-shell.el: ob-shell is the only babel language using a lambda to put
117 the execution function name together. */
118 pre.src-bash:before { content: 'bash'; }
119 pre.src-csh:before { content: 'csh'; }
120 pre.src-ash:before { content: 'ash'; }
121 pre.src-dash:before { content: 'dash'; }
122 pre.src-ksh:before { content: 'ksh'; }
123 pre.src-mksh:before { content: 'mksh'; }
124 pre.src-posh:before { content: 'posh'; }
125 /* Additional Emacs modes also supported by the LaTeX listings package */
126 pre.src-ada:before { content: 'Ada'; }
127 pre.src-asm:before { content: 'Assembler'; }
128 pre.src-caml:before { content: 'Caml'; }
129 pre.src-delphi:before { content: 'Delphi'; }
130 pre.src-html:before { content: 'HTML'; }
131 pre.src-idl:before { content: 'IDL'; }
132 pre.src-mercury:before { content: 'Mercury'; }
133 pre.src-metapost:before { content: 'MetaPost'; }
134 pre.src-modula-2:before { content: 'Modula-2'; }
135 pre.src-pascal:before { content: 'Pascal'; }
136 pre.src-ps:before { content: 'PostScript'; }
137 pre.src-prolog:before { content: 'Prolog'; }
138 pre.src-simula:before { content: 'Simula'; }
139 pre.src-tcl:before { content: 'tcl'; }
140 pre.src-tex:before { content: 'TeX'; }
141 pre.src-plain-tex:before { content: 'Plain TeX'; }
142 pre.src-verilog:before { content: 'Verilog'; }
143 pre.src-vhdl:before { content: 'VHDL'; }
144 pre.src-xml:before { content: 'XML'; }
145 pre.src-nxml:before { content: 'XML'; }
146 /* add a generic configuration mode; LaTeX export needs an additional
147 (add-to-list 'org-latex-listings-langs '(conf " ")) in .emacs */
148 pre.src-conf:before { content: 'Configuration File'; }
150 table { border-collapse:collapse; }
151 caption.t-above { caption-side: top; }
152 caption.t-bottom { caption-side: bottom; }
153 td, th { vertical-align:top; }
154 th.org-right { text-align: center; }
155 th.org-left { text-align: center; }
156 th.org-center { text-align: center; }
157 td.org-right { text-align: right; }
158 td.org-left { text-align: left; }
159 td.org-center { text-align: center; }
160 dt { font-weight: bold; }
161 .footpara { display: inline; }
162 .footdef { margin-bottom: 1em; }
163 .figure { padding: 1em; }
164 .figure p { text-align: center; }
165 .equation-container {
171 vertical-align: middle;
176 vertical-align: middle;
180 border: 2px solid gray;
185 { text-align: right; font-size: 70%; white-space: nowrap; }
186 textarea { overflow-x: auto; }
187 .linenr { font-size: smaller }
188 .code-highlighted { background-color: #ffff00; }
189 .org-info-js_info-navigation { border-style: none; }
190 #org-info-js_console-label
191 { font-size: 10px; font-weight: bold; white-space: nowrap; }
192 .org-info-js_search-highlight
193 { background-color: #ffff00; color: #000000; font-weight: bold; }
196 <link href="https://bootswatch.com/3/darkly/bootstrap.min.css" rel="stylesheet">
197 <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
198 <script src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.3.5/js/bootstrap.min.js"></script>
199 <style type="text/css">
200 footer {background-color: #111 !important;}
201 pre {background-color: #111; color: #ccc;}
203 <link rel="stylesheet" type="text/css" href="https://thomasf.github.io/solarized-css/solarized-dark.min.css" />
206 <div id="content" class="content">
207 <h1 class="title">Sixth Data - Data storage and computing engine</h1>
208 <div id="table-of-contents" role="doc-toc">
209 <h2>Table of Contents</h2>
210 <div id="text-table-of-contents" role="doc-toc">
212 <li><a href="#org575e4cf">1. General</a>
214 <li><a href="#org960610f">1.1. Source code</a></li>
217 <li><a href="#orga6ad9b1">2. Vision / goal</a></li>
218 <li><a href="#org923cf0c">3. Inspiration</a>
220 <li><a href="#org6e89e58">3.1. Brain</a></li>
221 <li><a href="#org75acc06">3.2. CM-1 Connection Machine</a></li>
224 <li><a href="#orgcce2d30">4. Reasons for hypercube as a so called first class citizen</a></li>
225 <li><a href="#orga5de57b">5. Geometrical computation idea</a>
227 <li><a href="#org7c36ce8">5.1. Distributed computation and data storage</a></li>
228 <li><a href="#orgb3e5e90">5.2. Mapping hypercube to object-oriented model and relational database</a></li>
229 <li><a href="#orgad3411a">5.3. Mapping entity relations in hypercube</a></li>
232 <li><a href="#orge747c35">6. Current status</a></li>
237 <div id="outline-container-org575e4cf" class="outline-2">
238 <h2 id="org575e4cf"><span class="section-number-2">1.</span> General</h2>
239 <div class="outline-text-2" id="text-1">
241 <li>This program is free software: released under Creative Commons Zero
246 <li>Svjatoslav Agejenko</li>
247 <li>Homepage: <a href="https://svjatoslav.eu">https://svjatoslav.eu</a></li>
248 <li>Email: <a href="mailto://svjatoslav@svjatoslav.eu">mailto://svjatoslav@svjatoslav.eu</a></li>
251 <li><a href="https://www.svjatoslav.eu/projects/">Other software projects hosted at svjatoslav.eu</a></li>
255 <div id="outline-container-org960610f" class="outline-3">
256 <h3 id="org960610f"><span class="section-number-3">1.1.</span> Source code</h3>
257 <div class="outline-text-3" id="text-1-1">
259 <li><a href="https://www2.svjatoslav.eu/gitweb/?p=sixth-data.git;a=snapshot;h=HEAD;sf=tgz">Download latest snapshot in TAR GZ format</a></li>
261 <li><a href="https://www2.svjatoslav.eu/gitweb/?p=sixth-data.git;a=summary">Browse Git repository online</a></li>
264 Clone Git repository using command:
266 <pre class="example">
267 git clone https://www2.svjatoslav.eu/git/sixth-data.git
270 <li>See <a href="https://www3.svjatoslav.eu/projects/sixth-data/apidocs/">JavaDoc</a>.</li>
276 <div id="outline-container-orga6ad9b1" class="outline-2">
277 <h2 id="orga6ad9b1"><span class="section-number-2">2.</span> Vision / goal</h2>
278 <div class="outline-text-2" id="text-2">
280 Provide hackable, versioned, optimized, distributed, geometrical,
281 arbitrary dimensional (<a href="#orgcce2d30">hypercube based</a>) data storage and computation
282 engine (<a href="#org6e89e58">as inspired by the brain</a>) for general purpose visual computing
283 environment called <a href="http://www2.svjatoslav.eu/gitbrowse/sixth/doc/index.html">Sixth</a>.
287 Because <a href="http://www.paulgraham.com/rootsoflisp.html">Lisp is hackable self defined programmable programming
288 language</a> it would be used to provide <a href="https://en.wikipedia.org/wiki/Imperative_programming">imperative programming</a> support.
292 <div id="outline-container-org923cf0c" class="outline-2">
293 <h2 id="org923cf0c"><span class="section-number-2">3.</span> Inspiration</h2>
294 <div class="outline-text-2" id="text-3">
296 <li>see also: <a href="https://en.wikipedia.org/wiki/OLAP_cube">OLAP cube</a>.</li>
299 <div id="outline-container-org6e89e58" class="outline-3">
300 <h3 id="org6e89e58"><span class="section-number-3">3.1.</span> Brain</h3>
301 <div class="outline-text-3" id="text-3-1">
303 <li>Brain appears to be natural geometrical/parallel data storage and
304 computational engine:
306 <li><a href="https://www.quantamagazine.org/the-brain-maps-out-ideas-and-memories-like-spaces-20190114/">https://www.quantamagazine.org/the-brain-maps-out-ideas-and-memories-like-spaces-20190114/</a></li>
307 <li><a href="https://www.simonsfoundation.org/2021/04/07/geometrical-thinking-offers-a-window-into-computation/">Geometrical Thinking Offers a Window Into Computation</a></li>
310 <li>Even more awesome is that brain appears to operate and is wired as
311 arbitrary/variable dimensional structure:
312 <a href="https://singularityhub.com/2017/06/21/is-there-a-multidimensional-mathematical-world-hidden-in-the-brains-computation/">https://singularityhub.com/2017/06/21/is-there-a-multidimensional-mathematical-world-hidden-in-the-brains-computation/</a></li>
314 <li>On top of this, this multidimensional space that brain represents
315 has dynamic/variable resolution/density:
317 <li><a href="https://www.quantamagazine.org/goals-and-rewards-redraw-the-brains-map-of-the-world-20190328">https://www.quantamagazine.org/goals-and-rewards-redraw-the-brains-map-of-the-world-20190328</a></li>
320 <li>Such properties allow parallel <a href="#orga5de57b">Geometrical computation</a> and
321 beautifully fits <a href="#org75acc06">CM-1 Connection Machine</a> architecture (for extra
322 hardware accelerated solution).</li>
328 <div id="outline-container-org75acc06" class="outline-3">
329 <h3 id="org75acc06"><span class="section-number-3">3.2.</span> CM-1 Connection Machine</h3>
330 <div class="outline-text-3" id="text-3-2">
332 <a href="https://en.wikipedia.org/wiki/Connection_Machine">https://en.wikipedia.org/wiki/Connection_Machine</a>
336 Massively parallel (thousands of CPUs) connected via
337 machine's internal 12-dimensional hypercube network allows to
338 efficiently simulate arbitrary dimensional hypercube and network
339 topology between computational units. So that when we are
340 solving/simulating for example 5 dimensional problem, we can arrange
341 computational units into virtual 5D network. See:
342 <a href="http://www.mission-base.com/tamiko/theory/cm_txts/di-ch2.html">http://www.mission-base.com/tamiko/theory/cm_txts/di-ch2.html</a>
346 we can pre-distribute data across computation units and perform
347 parallel <a href="#orga5de57b">geometrical computation</a>.
353 <div id="outline-container-orgcce2d30" class="outline-2">
354 <h2 id="orgcce2d30"><span class="section-number-2">4.</span> Reasons for hypercube as a so called first class citizen</h2>
355 <div class="outline-text-2" id="text-4">
357 <li>Hypercube is quite general purpose data structure that naturally
358 encapsulates wide variety data and problems.</li>
360 <li>Nicely captures apparent <a href="#org6e89e58">properties of the brain</a>.</li>
362 <li>Naturally supports distributed and parallel <a href="#orga5de57b">geometrical data storage
363 and computation.</a></li>
365 <li>Dedicated hardware like <a href="#org75acc06">CM-1</a> can be built around hypercube concept
366 that results in data, computation process and hardware, all
367 beautifully fitting together while complementing each other
370 <li>Hypercube stored data (and computation process) has geometry by its
371 nature and should fit nicely with "3D first" user interface ideology
372 of the parent <a href="http://www2.svjatoslav.eu/gitbrowse/sixth/doc/index.html">Sixth</a> project.</li>
376 <div id="outline-container-orga5de57b" class="outline-2">
377 <h2 id="orga5de57b"><span class="section-number-2">5.</span> Geometrical computation idea</h2>
378 <div class="outline-text-2" id="text-5">
380 <div id="outline-container-org7c36ce8" class="outline-3">
381 <h3 id="org7c36ce8"><span class="section-number-3">5.1.</span> Distributed computation and data storage</h3>
382 <div class="outline-text-3" id="text-5-1">
384 Lots of problems can be translated to geometry (use any shapes and as
385 many dimensions as you need). Solution(s) to such problems could be
386 then found via geometrical search/comparison/lookup results. As a
387 bonus, such geometrical <b>data storage</b> AND <b>computation</b> can be
388 naturally made in <b>parallel</b> and <b>distributed</b>.
392 Learning means building/updating/re-balancing the model (the hard
393 part). Question answering is making (relatively simple) lookups
394 (geometrical queries) against the model.
398 <div id="outline-container-orgb3e5e90" class="outline-3">
399 <h3 id="orgb3e5e90"><span class="section-number-3">5.2.</span> Mapping hypercube to object-oriented model and relational database</h3>
400 <div class="outline-text-3" id="text-5-2">
402 Object oriented programming is inspired by the way human mind
403 operates. It allows programmer to express ideas to computer in a more
408 It is actually also possible to map object model and relational
409 database to geometrical hyperspace:
413 <li>Object or database table row is a point in hypercube arbitrary
414 dimensional space. Each object member variable or database table
415 column can be mapped to its own dimension in hypercube. That is: if
416 class declares 4 variables for an object, then corresponding object
417 can be stored as a single point inside 4 dimensional
418 hypercube. Variable values translate to point coordinates in that
419 hypercube. That is: numbers and string can be translated to linear
420 value that can be used as a coordinate along particular dimension.</li>
422 <li>Each object class or database table declares its own hypercube that
423 contain instances (objects) of that class or rows of a table.</li>
428 <div id="outline-container-orgad3411a" class="outline-3">
429 <h3 id="orgad3411a"><span class="section-number-3">5.3.</span> Mapping entity relations in hypercube</h3>
430 <div class="outline-text-3" id="text-5-3">
432 Consider we want to create database of:
437 <li>Effort: Amount of time contributed by every author to every book
438 that he/she wrote.</li>
442 Information above can be represented as 3D cube where dimensions are:
451 Points in that cube would nicely capture many to many relations
452 between authors and the books.
457 <div id="outline-container-orge747c35" class="outline-2">
458 <h2 id="orge747c35"><span class="section-number-2">6.</span> Current status</h2>
459 <div class="outline-text-2" id="text-6">
461 <li>More or less defined <a href="#orga6ad9b1">Vision / goal</a>.</li>
463 <li>Collected some <a href="#org923cf0c">inspiring</a> <a href="#orga5de57b">ideas</a>.</li>
465 <li>Implemented very simple persistent key-value map.
467 <li>Long term goal is to use it as a backing storage engine and
468 implement more advanced features on top of this via layered
475 <div id="postamble" class="status">
476 <p class="author">Author: Svjatoslav Agejenko</p>
477 <p class="date">Created: 2022-10-30 Sun 12:09</p>
478 <p class="validation"><a href="https://validator.w3.org/check?uri=referer">Validate</a></p>