"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
-<!-- 2016-08-23 Tue 09:37 -->
-<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
-<meta name="viewport" content="width=device-width, initial-scale=1" />
-<title>Sixth - system for data storage, computation, exploration and interaction</title>
-<meta name="generator" content="Org-mode" />
-<meta name="author" content="Svjatoslav Agejenko" />
-<style type="text/css">
- <!--/*--><![CDATA[/*><!--*/
+<!-- 2022-10-30 Sun 12:09 -->
+<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
+<meta name="viewport" content="width=device-width, initial-scale=1" />
+<title>Sixth Data - Data storage and computing engine</title>
+<meta name="author" content="Svjatoslav Agejenko" />
+<meta name="generator" content="Org Mode" />
+<style>
+ #content { max-width: 60em; margin: auto; }
.title { text-align: center;
margin-bottom: .2em; }
.subtitle { text-align: center;
#postamble p, #preamble p { font-size: 90%; margin: .2em; }
p.verse { margin-left: 3%; }
pre {
- border: 1px solid #ccc;
- box-shadow: 3px 3px 3px #eee;
+ border: 1px solid #e6e6e6;
+ border-radius: 3px;
+ background-color: #f2f2f2;
padding: 8pt;
font-family: monospace;
overflow: auto;
}
pre.src {
position: relative;
- overflow: visible;
- padding-top: 1.2em;
+ overflow: auto;
}
pre.src:before {
display: none;
position: absolute;
- background-color: white;
- top: -10px;
- right: 10px;
+ top: -8px;
+ right: 12px;
padding: 3px;
- border: 1px solid black;
+ color: #555;
+ background-color: #f2f2f299;
}
- pre.src:hover:before { display: inline;}
- pre.src-sh:before { content: 'sh'; }
- pre.src-bash:before { content: 'sh'; }
+ pre.src:hover:before { display: inline; margin-top: 14px;}
+ /* Languages per Org manual */
+ pre.src-asymptote:before { content: 'Asymptote'; }
+ pre.src-awk:before { content: 'Awk'; }
+ pre.src-authinfo::before { content: 'Authinfo'; }
+ pre.src-C:before { content: 'C'; }
+ /* pre.src-C++ doesn't work in CSS */
+ pre.src-clojure:before { content: 'Clojure'; }
+ pre.src-css:before { content: 'CSS'; }
+ pre.src-D:before { content: 'D'; }
+ pre.src-ditaa:before { content: 'ditaa'; }
+ pre.src-dot:before { content: 'Graphviz'; }
+ pre.src-calc:before { content: 'Emacs Calc'; }
pre.src-emacs-lisp:before { content: 'Emacs Lisp'; }
- pre.src-R:before { content: 'R'; }
- pre.src-perl:before { content: 'Perl'; }
- pre.src-java:before { content: 'Java'; }
- pre.src-sql:before { content: 'SQL'; }
+ pre.src-fortran:before { content: 'Fortran'; }
+ pre.src-gnuplot:before { content: 'gnuplot'; }
+ pre.src-haskell:before { content: 'Haskell'; }
+ pre.src-hledger:before { content: 'hledger'; }
+ pre.src-java:before { content: 'Java'; }
+ pre.src-js:before { content: 'Javascript'; }
+ pre.src-latex:before { content: 'LaTeX'; }
+ pre.src-ledger:before { content: 'Ledger'; }
+ pre.src-lisp:before { content: 'Lisp'; }
+ pre.src-lilypond:before { content: 'Lilypond'; }
+ pre.src-lua:before { content: 'Lua'; }
+ pre.src-matlab:before { content: 'MATLAB'; }
+ pre.src-mscgen:before { content: 'Mscgen'; }
+ pre.src-ocaml:before { content: 'Objective Caml'; }
+ pre.src-octave:before { content: 'Octave'; }
+ pre.src-org:before { content: 'Org mode'; }
+ pre.src-oz:before { content: 'OZ'; }
+ pre.src-plantuml:before { content: 'Plantuml'; }
+ pre.src-processing:before { content: 'Processing.js'; }
+ pre.src-python:before { content: 'Python'; }
+ pre.src-R:before { content: 'R'; }
+ pre.src-ruby:before { content: 'Ruby'; }
+ pre.src-sass:before { content: 'Sass'; }
+ pre.src-scheme:before { content: 'Scheme'; }
+ pre.src-screen:before { content: 'Gnu Screen'; }
+ pre.src-sed:before { content: 'Sed'; }
+ pre.src-sh:before { content: 'shell'; }
+ pre.src-sql:before { content: 'SQL'; }
+ pre.src-sqlite:before { content: 'SQLite'; }
+ /* additional languages in org.el's org-babel-load-languages alist */
+ pre.src-forth:before { content: 'Forth'; }
+ pre.src-io:before { content: 'IO'; }
+ pre.src-J:before { content: 'J'; }
+ pre.src-makefile:before { content: 'Makefile'; }
+ pre.src-maxima:before { content: 'Maxima'; }
+ pre.src-perl:before { content: 'Perl'; }
+ pre.src-picolisp:before { content: 'Pico Lisp'; }
+ pre.src-scala:before { content: 'Scala'; }
+ pre.src-shell:before { content: 'Shell Script'; }
+ pre.src-ebnf2ps:before { content: 'ebfn2ps'; }
+ /* additional language identifiers per "defun org-babel-execute"
+ in ob-*.el */
+ pre.src-cpp:before { content: 'C++'; }
+ pre.src-abc:before { content: 'ABC'; }
+ pre.src-coq:before { content: 'Coq'; }
+ pre.src-groovy:before { content: 'Groovy'; }
+ /* additional language identifiers from org-babel-shell-names in
+ ob-shell.el: ob-shell is the only babel language using a lambda to put
+ the execution function name together. */
+ pre.src-bash:before { content: 'bash'; }
+ pre.src-csh:before { content: 'csh'; }
+ pre.src-ash:before { content: 'ash'; }
+ pre.src-dash:before { content: 'dash'; }
+ pre.src-ksh:before { content: 'ksh'; }
+ pre.src-mksh:before { content: 'mksh'; }
+ pre.src-posh:before { content: 'posh'; }
+ /* Additional Emacs modes also supported by the LaTeX listings package */
+ pre.src-ada:before { content: 'Ada'; }
+ pre.src-asm:before { content: 'Assembler'; }
+ pre.src-caml:before { content: 'Caml'; }
+ pre.src-delphi:before { content: 'Delphi'; }
+ pre.src-html:before { content: 'HTML'; }
+ pre.src-idl:before { content: 'IDL'; }
+ pre.src-mercury:before { content: 'Mercury'; }
+ pre.src-metapost:before { content: 'MetaPost'; }
+ pre.src-modula-2:before { content: 'Modula-2'; }
+ pre.src-pascal:before { content: 'Pascal'; }
+ pre.src-ps:before { content: 'PostScript'; }
+ pre.src-prolog:before { content: 'Prolog'; }
+ pre.src-simula:before { content: 'Simula'; }
+ pre.src-tcl:before { content: 'tcl'; }
+ pre.src-tex:before { content: 'TeX'; }
+ pre.src-plain-tex:before { content: 'Plain TeX'; }
+ pre.src-verilog:before { content: 'Verilog'; }
+ pre.src-vhdl:before { content: 'VHDL'; }
+ pre.src-xml:before { content: 'XML'; }
+ pre.src-nxml:before { content: 'XML'; }
+ /* add a generic configuration mode; LaTeX export needs an additional
+ (add-to-list 'org-latex-listings-langs '(conf " ")) in .emacs */
+ pre.src-conf:before { content: 'Configuration File'; }
table { border-collapse:collapse; }
caption.t-above { caption-side: top; }
.footdef { margin-bottom: 1em; }
.figure { padding: 1em; }
.figure p { text-align: center; }
+ .equation-container {
+ display: table;
+ text-align: center;
+ width: 100%;
+ }
+ .equation {
+ vertical-align: middle;
+ }
+ .equation-label {
+ display: table-cell;
+ text-align: right;
+ vertical-align: middle;
+ }
.inlinetask {
padding: 10px;
border: 2px solid gray;
{ font-size: 10px; font-weight: bold; white-space: nowrap; }
.org-info-js_search-highlight
{ background-color: #ffff00; color: #000000; font-weight: bold; }
- /*]]>*/-->
+ .org-svg { }
+</style>
+<link href="https://bootswatch.com/3/darkly/bootstrap.min.css" rel="stylesheet">
+<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
+<script src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.3.5/js/bootstrap.min.js"></script>
+<style type="text/css">
+footer {background-color: #111 !important;}
+pre {background-color: #111; color: #ccc;}
</style>
-<link rel="stylesheet" type="text/css" href="http://thomasf.github.io/solarized-css/solarized-dark.min.css" />
-<script type="text/javascript">
-/*
-@licstart The following is the entire license notice for the
-JavaScript code in this tag.
-
-Copyright (C) 2012-2013 Free Software Foundation, Inc.
-
-The JavaScript code in this tag is free software: you can
-redistribute it and/or modify it under the terms of the GNU
-General Public License (GNU GPL) as published by the Free Software
-Foundation, either version 3 of the License, or (at your option)
-any later version. The code is distributed WITHOUT ANY WARRANTY;
-without even the implied warranty of MERCHANTABILITY or FITNESS
-FOR A PARTICULAR PURPOSE. See the GNU GPL for more details.
-
-As additional permission under GNU GPL version 3 section 7, you
-may distribute non-source (e.g., minimized or compacted) forms of
-that code without the copy of the GNU GPL normally required by
-section 4, provided you include this license notice and a URL
-through which recipients can access the Corresponding Source.
-
-
-@licend The above is the entire license notice
-for the JavaScript code in this tag.
-*/
-<!--/*--><![CDATA[/*><!--*/
- function CodeHighlightOn(elem, id)
- {
- var target = document.getElementById(id);
- if(null != target) {
- elem.cacheClassElem = elem.className;
- elem.cacheClassTarget = target.className;
- target.className = "code-highlighted";
- elem.className = "code-highlighted";
- }
- }
- function CodeHighlightOff(elem, id)
- {
- var target = document.getElementById(id);
- if(elem.cacheClassElem)
- elem.className = elem.cacheClassElem;
- if(elem.cacheClassTarget)
- target.className = elem.cacheClassTarget;
- }
-/*]]>*///-->
-</script>
+<link rel="stylesheet" type="text/css" href="https://thomasf.github.io/solarized-css/solarized-dark.min.css" />
</head>
<body>
-<div id="content">
-<h1 class="title">Sixth - system for data storage, computation, exploration and interaction</h1>
-<div id="table-of-contents">
+<div id="content" class="content">
+<h1 class="title">Sixth Data - Data storage and computing engine</h1>
+<div id="table-of-contents" role="doc-toc">
<h2>Table of Contents</h2>
-<div id="text-table-of-contents">
+<div id="text-table-of-contents" role="doc-toc">
<ul>
-<li><a href="#orgheadline1">1. Current status</a></li>
+<li><a href="#org575e4cf">1. General</a>
+<ul>
+<li><a href="#org960610f">1.1. Source code</a></li>
+</ul>
+</li>
+<li><a href="#orga6ad9b1">2. Vision / goal</a></li>
+<li><a href="#org923cf0c">3. Inspiration</a>
+<ul>
+<li><a href="#org6e89e58">3.1. Brain</a></li>
+<li><a href="#org75acc06">3.2. CM-1 Connection Machine</a></li>
+</ul>
+</li>
+<li><a href="#orgcce2d30">4. Reasons for hypercube as a so called first class citizen</a></li>
+<li><a href="#orga5de57b">5. Geometrical computation idea</a>
+<ul>
+<li><a href="#org7c36ce8">5.1. Distributed computation and data storage</a></li>
+<li><a href="#orgb3e5e90">5.2. Mapping hypercube to object-oriented model and relational database</a></li>
+<li><a href="#orgad3411a">5.3. Mapping entity relations in hypercube</a></li>
+</ul>
+</li>
+<li><a href="#orge747c35">6. Current status</a></li>
</ul>
</div>
</div>
-<hr />
-<ul class="org-ul">
-<li>This is a subproject of <a href="http://www2.svjatoslav.eu/gitbrowse/sixth/doc/index.html">Sixth</a></li>
-<li><a href="http://www2.svjatoslav.eu/gitweb/?p=sixth-data.git;a=snapshot;h=HEAD;sf=tgz">download latest snapshot</a></li>
-
-<li>This program is free software; you can redistribute it and/or modify
-it under the terms of version 3 of the <a href="https://www.gnu.org/licenses/lgpl.html">GNU Lesser General Public
-License</a> or later as published by the Free Software Foundation.</li>
+<div id="outline-container-org575e4cf" class="outline-2">
+<h2 id="org575e4cf"><span class="section-number-2">1.</span> General</h2>
+<div class="outline-text-2" id="text-1">
+<ul class="org-ul">
+<li>This program is free software: released under Creative Commons Zero
+(CC0) license</li>
<li>Program author:
<ul class="org-ul">
<li>Svjatoslav Agejenko</li>
-<li>Homepage: <a href="http://svjatoslav.eu/">http://svjatoslav.eu/</a></li>
-<li>Email: <a href="mailto://svjatoslav@svjatoslav.eu/">mailto://svjatoslav@svjatoslav.eu/</a></li>
+<li>Homepage: <a href="https://svjatoslav.eu">https://svjatoslav.eu</a></li>
+<li>Email: <a href="mailto://svjatoslav@svjatoslav.eu">mailto://svjatoslav@svjatoslav.eu</a></li>
</ul></li>
-<li><a href="http://svjatoslav.eu/programs.jsp">other applications hosted at svjatoslav.eu</a></li>
+<li><a href="https://www.svjatoslav.eu/projects/">Other software projects hosted at svjatoslav.eu</a></li>
</ul>
+</div>
+<div id="outline-container-org960610f" class="outline-3">
+<h3 id="org960610f"><span class="section-number-3">1.1.</span> Source code</h3>
+<div class="outline-text-3" id="text-1-1">
+<ul class="org-ul">
+<li><a href="https://www2.svjatoslav.eu/gitweb/?p=sixth-data.git;a=snapshot;h=HEAD;sf=tgz">Download latest snapshot in TAR GZ format</a></li>
-<div id="outline-container-orgheadline1" class="outline-2">
-<h2 id="orgheadline1"><span class="section-number-2">1</span> Current status</h2>
-<div class="outline-text-2" id="text-1">
+<li><a href="https://www2.svjatoslav.eu/gitweb/?p=sixth-data.git;a=summary">Browse Git repository online</a></li>
+
+<li><p>
+Clone Git repository using command:
+</p>
+<pre class="example">
+git clone https://www2.svjatoslav.eu/git/sixth-data.git
+</pre></li>
+
+<li>See <a href="https://www3.svjatoslav.eu/projects/sixth-data/apidocs/">JavaDoc</a>.</li>
+</ul>
+</div>
+</div>
+</div>
+
+<div id="outline-container-orga6ad9b1" class="outline-2">
+<h2 id="orga6ad9b1"><span class="section-number-2">2.</span> Vision / goal</h2>
+<div class="outline-text-2" id="text-2">
+<p>
+Provide hackable, versioned, optimized, distributed, geometrical,
+arbitrary dimensional (<a href="#orgcce2d30">hypercube based</a>) data storage and computation
+engine (<a href="#org6e89e58">as inspired by the brain</a>) for general purpose visual computing
+environment called <a href="http://www2.svjatoslav.eu/gitbrowse/sixth/doc/index.html">Sixth</a>.
+</p>
+
+<p>
+Because <a href="http://www.paulgraham.com/rootsoflisp.html">Lisp is hackable self defined programmable programming
+language</a> it would be used to provide <a href="https://en.wikipedia.org/wiki/Imperative_programming">imperative programming</a> support.
+</p>
+</div>
+</div>
+<div id="outline-container-org923cf0c" class="outline-2">
+<h2 id="org923cf0c"><span class="section-number-2">3.</span> Inspiration</h2>
+<div class="outline-text-2" id="text-3">
<ul class="org-ul">
-<li>Implemented very simple persistent key-value map.</li>
+<li>see also: <a href="https://en.wikipedia.org/wiki/OLAP_cube">OLAP cube</a>.</li>
</ul>
+</div>
+<div id="outline-container-org6e89e58" class="outline-3">
+<h3 id="org6e89e58"><span class="section-number-3">3.1.</span> Brain</h3>
+<div class="outline-text-3" id="text-3-1">
+<ul class="org-ul">
+<li>Brain appears to be natural geometrical/parallel data storage and
+computational engine:
+<ul class="org-ul">
+<li><a href="https://www.quantamagazine.org/the-brain-maps-out-ideas-and-memories-like-spaces-20190114/">https://www.quantamagazine.org/the-brain-maps-out-ideas-and-memories-like-spaces-20190114/</a></li>
+<li><a href="https://www.simonsfoundation.org/2021/04/07/geometrical-thinking-offers-a-window-into-computation/">Geometrical Thinking Offers a Window Into Computation</a></li>
+</ul></li>
+
+<li>Even more awesome is that brain appears to operate and is wired as
+arbitrary/variable dimensional structure:
+<a href="https://singularityhub.com/2017/06/21/is-there-a-multidimensional-mathematical-world-hidden-in-the-brains-computation/">https://singularityhub.com/2017/06/21/is-there-a-multidimensional-mathematical-world-hidden-in-the-brains-computation/</a></li>
+
+<li>On top of this, this multidimensional space that brain represents
+has dynamic/variable resolution/density:
+<ul class="org-ul">
+<li><a href="https://www.quantamagazine.org/goals-and-rewards-redraw-the-brains-map-of-the-world-20190328">https://www.quantamagazine.org/goals-and-rewards-redraw-the-brains-map-of-the-world-20190328</a></li>
+</ul></li>
+
+<li>Such properties allow parallel <a href="#orga5de57b">Geometrical computation</a> and
+beautifully fits <a href="#org75acc06">CM-1 Connection Machine</a> architecture (for extra
+hardware accelerated solution).</li>
+</ul>
+</div>
+</div>
+
+
+<div id="outline-container-org75acc06" class="outline-3">
+<h3 id="org75acc06"><span class="section-number-3">3.2.</span> CM-1 Connection Machine</h3>
+<div class="outline-text-3" id="text-3-2">
+<p>
+<a href="https://en.wikipedia.org/wiki/Connection_Machine">https://en.wikipedia.org/wiki/Connection_Machine</a>
+</p>
+
+<p>
+Massively parallel (thousands of CPUs) connected via
+machine's internal 12-dimensional hypercube network allows to
+efficiently simulate arbitrary dimensional hypercube and network
+topology between computational units. So that when we are
+solving/simulating for example 5 dimensional problem, we can arrange
+computational units into virtual 5D network. See:
+<a href="http://www.mission-base.com/tamiko/theory/cm_txts/di-ch2.html">http://www.mission-base.com/tamiko/theory/cm_txts/di-ch2.html</a>
+</p>
<p>
-Long term goal is to implement more advanced features on top of this.
+we can pre-distribute data across computation units and perform
+parallel <a href="#orga5de57b">geometrical computation</a>.
</p>
</div>
</div>
</div>
+
+<div id="outline-container-orgcce2d30" class="outline-2">
+<h2 id="orgcce2d30"><span class="section-number-2">4.</span> Reasons for hypercube as a so called first class citizen</h2>
+<div class="outline-text-2" id="text-4">
+<ul class="org-ul">
+<li>Hypercube is quite general purpose data structure that naturally
+encapsulates wide variety data and problems.</li>
+
+<li>Nicely captures apparent <a href="#org6e89e58">properties of the brain</a>.</li>
+
+<li>Naturally supports distributed and parallel <a href="#orga5de57b">geometrical data storage
+and computation.</a></li>
+
+<li>Dedicated hardware like <a href="#org75acc06">CM-1</a> can be built around hypercube concept
+that results in data, computation process and hardware, all
+beautifully fitting together while complementing each other
+strengths.</li>
+
+<li>Hypercube stored data (and computation process) has geometry by its
+nature and should fit nicely with "3D first" user interface ideology
+of the parent <a href="http://www2.svjatoslav.eu/gitbrowse/sixth/doc/index.html">Sixth</a> project.</li>
+</ul>
+</div>
+</div>
+<div id="outline-container-orga5de57b" class="outline-2">
+<h2 id="orga5de57b"><span class="section-number-2">5.</span> Geometrical computation idea</h2>
+<div class="outline-text-2" id="text-5">
+</div>
+<div id="outline-container-org7c36ce8" class="outline-3">
+<h3 id="org7c36ce8"><span class="section-number-3">5.1.</span> Distributed computation and data storage</h3>
+<div class="outline-text-3" id="text-5-1">
+<p>
+Lots of problems can be translated to geometry (use any shapes and as
+many dimensions as you need). Solution(s) to such problems could be
+then found via geometrical search/comparison/lookup results. As a
+bonus, such geometrical <b>data storage</b> AND <b>computation</b> can be
+naturally made in <b>parallel</b> and <b>distributed</b>.
+</p>
+
+<p>
+Learning means building/updating/re-balancing the model (the hard
+part). Question answering is making (relatively simple) lookups
+(geometrical queries) against the model.
+</p>
+</div>
+</div>
+<div id="outline-container-orgb3e5e90" class="outline-3">
+<h3 id="orgb3e5e90"><span class="section-number-3">5.2.</span> Mapping hypercube to object-oriented model and relational database</h3>
+<div class="outline-text-3" id="text-5-2">
+<p>
+Object oriented programming is inspired by the way human mind
+operates. It allows programmer to express ideas to computer in a more
+human-like terms.
+</p>
+
+<p>
+It is actually also possible to map object model and relational
+database to geometrical hyperspace:
+</p>
+
+<ul class="org-ul">
+<li>Object or database table row is a point in hypercube arbitrary
+dimensional space. Each object member variable or database table
+column can be mapped to its own dimension in hypercube. That is: if
+class declares 4 variables for an object, then corresponding object
+can be stored as a single point inside 4 dimensional
+hypercube. Variable values translate to point coordinates in that
+hypercube. That is: numbers and string can be translated to linear
+value that can be used as a coordinate along particular dimension.</li>
+
+<li>Each object class or database table declares its own hypercube that
+contain instances (objects) of that class or rows of a table.</li>
+</ul>
+</div>
+</div>
+
+<div id="outline-container-orgad3411a" class="outline-3">
+<h3 id="orgad3411a"><span class="section-number-3">5.3.</span> Mapping entity relations in hypercube</h3>
+<div class="outline-text-3" id="text-5-3">
+<p>
+Consider we want to create database of:
+</p>
+<ul class="org-ul">
+<li>Books.</li>
+<li>Authors.</li>
+<li>Effort: Amount of time contributed by every author to every book
+that he/she wrote.</li>
+</ul>
+
+<p>
+Information above can be represented as 3D cube where dimensions are:
+</p>
+<ul class="org-ul">
+<li>X: Book</li>
+<li>Y: Author</li>
+<li>Z: Effort</li>
+</ul>
+
+<p>
+Points in that cube would nicely capture many to many relations
+between authors and the books.
+</p>
+</div>
+</div>
+</div>
+<div id="outline-container-orge747c35" class="outline-2">
+<h2 id="orge747c35"><span class="section-number-2">6.</span> Current status</h2>
+<div class="outline-text-2" id="text-6">
+<ul class="org-ul">
+<li>More or less defined <a href="#orga6ad9b1">Vision / goal</a>.</li>
+
+<li>Collected some <a href="#org923cf0c">inspiring</a> <a href="#orga5de57b">ideas</a>.</li>
+
+<li>Implemented very simple persistent key-value map.
+<ul class="org-ul">
+<li>Long term goal is to use it as a backing storage engine and
+implement more advanced features on top of this via layered
+architecture.</li>
+</ul></li>
+</ul>
+</div>
+</div>
+</div>
<div id="postamble" class="status">
<p class="author">Author: Svjatoslav Agejenko</p>
-<p class="date">Created: 2016-08-23 Tue 09:37</p>
-<p class="validation"><a href="http://validator.w3.org/check?uri=referer">Validate</a></p>
+<p class="date">Created: 2022-10-30 Sun 12:09</p>
+<p class="validation"><a href="https://validator.w3.org/check?uri=referer">Validate</a></p>
</div>
</body>
</html>
projects / sixth-data.git / blobdiff