X-Git-Url: http://www2.svjatoslav.eu/gitweb/?a=blobdiff_plain;f=doc%2Findex.html;h=b6bc9d1621830360b1e65fd0f6bfa84a55d6e729;hb=c20ef78b553cad795177f4415a20ba8f581426b9;hp=c719c14e1a95c91702ffdb0a7e7b8693eb7925cd;hpb=b382105757c3d9a2bb528fa8e4218354feff2429;p=sixth-data.git diff --git a/doc/index.html b/doc/index.html index c719c14..b6bc9d1 100644 --- a/doc/index.html +++ b/doc/index.html @@ -1,346 +1,432 @@ - - + + + -Sixth - system for data storage, computation, exploration and interaction - - - - - - - -" + + + +Sixth Data - Data storage and computing engine + + - + + + + + -
-

Sixth - system for data storage, computation, exploration and interaction

-
-
+
-
-

2 Inspiration

+
+

2 Vision / goal

+

+Provide hackable, versioned, optimized, distributed, geometrical, +arbitrary dimensional (hypercube based) data storage and computation +engine (as inspired by the brain) for general purpose visual computing +environment called Sixth. +

+ +

+Because Lisp is hackable self defined programmable programming +language it would be used to provide imperative programming support. +

+
+
+
+

3 Inspiration

+
    -
  • Relational databases: -
      -
    • Transactional. -
      • -
    • Indexable / Quickly searchable. -
      • +
    • see also: OLAP cube.
    -
    • +
+
+

3.1 Brain

+
+ +
+
+ + +
+

3.2 CM-1 Connection Machine

+

-https://www.kenneth-truyers.net/2016/10/13/git-nosql-database/ ) +https://en.wikipedia.org/wiki/Connection_Machine +

+ +

+Massively parallel (thousands of CPUs) connected via +machine's internal 12-dimensional hypercube network allows to +efficiently simulate arbitrary dimensional hypercube and network +topology between computational units. So that when we are +solving/simulating for example 5 dimensional problem, we can arrange +computational units into virtual 5D network. See: +http://www.mission-base.com/tamiko/theory/cm_txts/di-ch2.html +

+ +

+we can pre-distribute data across computation units and perform +parallel geometrical computation.

- - +
+
-
-

2.1 Brain

-
-
+
+

5 Geometrical computation idea

+
+
+

5.1 Distributed computation and data storage

+
+

+Lots of problems can be translated to geometry (use any shapes and as +many dimensions as you need). Solution(s) to such problems could be +then found via geometrical search/comparison/lookup results. As a +bonus, such geometrical data storage AND computation can be +naturally made in parallel and distributed. +

-
-

3 Ideas

-
-
-

3.1 Distributed computation and data storage

-

-Maybe every problem can be translated to geometry (use any shapes and -as many dimensions as you need). Solution(s) to such problems would -then appear as relatively simple search/comparison/lookup results. As -a bonus, such geometrical *data storage* AND *computation* can be -naturally made in *parallel* and *distributed*. That's what neurons in -the brain appear to be doing ! :) . Learning means building/updating -the model (the hard part). Question answering is making (relatively -simple) lookups (geometrical queries) against the model. +Learning means building/updating/re-balancing the model (the hard +part). Question answering is making (relatively simple) lookups +(geometrical queries) against the model.

-
-

3.2 Mapping of hyperspace to traditional object-oriented model

-
+
+

5.2 Mapping hypercube to object-oriented model and relational database

+

Object oriented programming is inspired by the way human mind operates. It allows programmer to express ideas to computer in a more @@ -348,164 +434,170 @@ human-like terms.

-It is possible to map object model to geometrical hyperspace: +It is actually also possible to map object model and relational +database to geometrical hyperspace:

    -
  • Object is a point in space (universe). Each object member variable -translates to its own dimension. That is: if class declares 4 -variables for an object, then corresponding object can be stored as -a single point inside 4 dimensional space. Variable values translate -to point coordinates in space. That is: Integer, floating point -number and even boolean and string can be translated to linear value -that can be used as a coordinate along particular dimension. -
    • - -
  • Each class declares its own space (universe). All class instances -(objects) are points inside that particular universe. References -between objects of different types are hyperlinks (portals) between -different universes. -
    • +
  • Object or database table row is a point in hypercube arbitrary +dimensional space. Each object member variable or database table +column can be mapped to its own dimension in hypercube. That is: if +class declares 4 variables for an object, then corresponding object +can be stored as a single point inside 4 dimensional +hypercube. Variable values translate to point coordinates in that +hypercube. That is: numbers and string can be translated to linear +value that can be used as a coordinate along particular dimension.
    • + +
  • Each object class or database table declares its own hypercube that +contain instances (objects) of that class or rows of a table.
-
-

3.3 Handling of relations

-
+ +
+

5.3 Mapping entity relations in hypercube

+

-Consider we want to create database of books and authors. Book can -have multiple authors, and single person can be author for multiple -books. It is possible to store how many hours of work each author has -contributed to every book, using hyperspace as follows: +Consider we want to create database of:

- -
    -
  • Every dimension corresponds to one particular book author. (10 -authors in the database, would require 10 dimensional space) -
      -
    • Point in space corresponds to one particular book.
        -
      • Point location along particular (author) dimension corresponds -to amount of work contributed by particular author for given -book. -
        • -
      -
      • -
    -
    • +
  • Books.
    • +
  • Authors.
    • +
  • Effort: Amount of time contributed by every author to every book +that he/she wrote.

-Alternatively: +Information above can be represented as 3D cube where dimensions are:

- -
    -
  • Every dimension corresponds to one particular book. -
      -
    • Point in space corresponds to one particular author in the entire -database.
        -
      • Point location along particular (book) dimension corresponds to -amount of work contributed for book by given author (point). -
        • -
      -
      • -
    -
    • +
  • X: Book
    • +
  • Y: Author
    • +
  • Z: Effort
+ +

+Points in that cube would nicely capture many to many relations +between authors and the books. +

- - -
-

4 Current status

-
+
+

6 Current status

+
    -
  • More or less defined Vision / goal. -
    • +
  • More or less defined Vision / goal.
    • -
  • Collected some ideas. -
    • +
  • Collected some inspiring ideas.
  • Implemented very simple persistent key-value map.
    • Long term goal is to use it as a backing storage engine and -implement more advanced features on top of this. -
      • -
    -
    • +implement more advanced features on top of this via layered +architecture. +
-
-

5 See also

-
+
+

7 See also

+

Interesting or competing projects with good ideas:

    -
  • GRAKN.AI: database in the form of a knowledge graph that uses -machine reasoning to simplify data processing challenges for AI -applications. +
  • flexible user interface building for interacting with different data
      -
    • https://grakn.ai/ -
      • +
    • Glamorous Toolkit +
        +
      • Moldable development environment. It is a live notebook. It is a +flexible search interface. It is a fancy code editor. It is a +software analysis platform. It is a data visualization engine. All +in one.
        • +
      • +
- +
-
  • Gemstone/S based on Smalltalk. +
    • +
  • Array Databases: Concepts, Standards, Implementations
    • -
  • Magma distributed database in Smalltalk. +
  • TileDB
    • + +
  • CM-1 Connection Machine
    • + +
  • Lisp-Stat: An environment for Statistical Computing
    • - +
    +
    -
  • ZetaVM +
    +

    7.2 Distributed, reliable, parallel computing systems

    +
    -
    • +
  • ChrysaLisp +
      +
    • Assembler/C-Script/Lisp 64 bit, MIMD, multi CPU, multi threaded, +multi core, multi user Parallel OS. With GUI, Terminal, OO +Assembler, Class libraries, C-Script compiler, Lisp interpreter, +Debugger, and more…
      • +
    • + +
  • Gemstone/S +
      +
    • Completely distributed smalltalk based computing +system.
      • +
    • + +
  • http://phantomos.org/ +
      +
    • Programs run forever. System crash or reboot does not destroy +state of running program.
      • +
    • + +
  • Magma +
      +
    • Multi-user object database for Squeak
      • +
    • + +
  • TAOS +
      +
    • Completely distributed operating system/virtual machine:
      • +
    • -
    -
    -
    -

    Author: Svjatoslav Agejenko

    -

    Created: 2017-07-29 Sat 13:31

    -

    Emacs 25.1.1 (Org-mode 8.2.10)

    -
    +
    +
    +
    +

    Author: Svjatoslav Agejenko

    +

    Created: 2021-04-09 Fri 16:55

    +

    Validate

    +