X-Git-Url: http://www2.svjatoslav.eu/gitweb/?a=blobdiff_plain;f=doc%2Findex.html;h=b6bc9d1621830360b1e65fd0f6bfa84a55d6e729;hb=c20ef78b553cad795177f4415a20ba8f581426b9;hp=0c9559e4a8b4b6c9c002cd8815be43cff136c11e;hpb=c2e5bc51afcb4776720ff274596bd68595f793d7;p=sixth-data.git diff --git a/doc/index.html b/doc/index.html index 0c9559e..b6bc9d1 100644 --- a/doc/index.html +++ b/doc/index.html @@ -1,364 +1,603 @@ - - + + + -Sixth - system for data storage, computation, exploration and interaction - - - - - - - -" + + + +Sixth Data - Data storage and computing engine + + - + + + + + -
-

Sixth - system for data storage, computation, exploration and interaction

-
- +
+
+

1.1 Source code

+
+ +
+
+
+ +
+

2 Vision / goal

+

-Provide versioned, clustered, flexible, distributed, multi-dimensional -data storage engine for the Sixth computation engine. +Provide hackable, versioned, optimized, distributed, geometrical, +arbitrary dimensional (hypercube based) data storage and computation +engine (as inspired by the brain) for general purpose visual computing +environment called Sixth.

+

+Because Lisp is hackable self defined programmable programming +language it would be used to provide imperative programming support. +

+
+
+
+

3 Inspiration

+
    -
  • Speaking of traditional relational database and object oriented -business applications: +
  • see also: OLAP cube.
    • +
+
+
+

3.1 Brain

+
- +
  • https://www.quantamagazine.org/goals-and-rewards-redraw-the-brains-map-of-the-world-20190328
    • + + +
  • Such properties allow parallel Geometrical computation and +beautifully fits CM-1 Connection Machine architecture (for extra +hardware accelerated solution).
    • +
    -
    -

    1.1 Inspiration

    -
    -
    + +
    +

    4 Reasons for hypercube as a so called first class citizen

    +
      -
    • Transactional. -
      • -
    • Indexable / Quickly searchable. -
      • +
    • Hypercube is quite general purpose data structure that naturally +encapsulates wide variety data and problems.
      • + +
    • Nicely captures apparent properties of the brain.
      • + +
    • Naturally supports distributed and parallel geometrical data storage +and computation.
      • + +
    • Dedicated hardware like CM-1 can be built around hypercube concept +that results in data, computation process and hardware, all +beautifully fitting together while complementing each other +strengths.
      • + +
    • Hypercube stored data (and computation process) has geometry by its +nature and should fit nicely with "3D first" user interface ideology +of the parent Sixth project.
    - +
    +
    +
    +

    5 Geometrical computation idea

    +
    +
    +
    +

    5.1 Distributed computation and data storage

    +
    +

    +Lots of problems can be translated to geometry (use any shapes and as +many dimensions as you need). Solution(s) to such problems could be +then found via geometrical search/comparison/lookup results. As a +bonus, such geometrical data storage AND computation can be +naturally made in parallel and distributed. +

    + +

    +Learning means building/updating/re-balancing the model (the hard +part). Question answering is making (relatively simple) lookups +(geometrical queries) against the model. +

    +
    +
    +
    +

    5.2 Mapping hypercube to object-oriented model and relational database

    +
    +

    +Object oriented programming is inspired by the way human mind +operates. It allows programmer to express ideas to computer in a more +human-like terms. +

    + +

    +It is actually also possible to map object model and relational +database to geometrical hyperspace: +

    -
  • Git (version control system)
      -
    • Versionable -
      • -
    • Branchable / mergeable. -
      • -
    • Transparent cansistency, checksumming and deduplication. -
      • -
    • (Git as a database: -https://www.kenneth-truyers.net/2016/10/13/git-nosql-database/ ) -
      • +
    • Object or database table row is a point in hypercube arbitrary +dimensional space. Each object member variable or database table +column can be mapped to its own dimension in hypercube. That is: if +class declares 4 variables for an object, then corresponding object +can be stored as a single point inside 4 dimensional +hypercube. Variable values translate to point coordinates in that +hypercube. That is: numbers and string can be translated to linear +value that can be used as a coordinate along particular dimension.
      • + +
    • Each object class or database table declares its own hypercube that +contain instances (objects) of that class or rows of a table.
    -
    • +
    +
    -
  • Brain (appears to have more than 3D dimensional design. Food for -thought…) -
      -
    • https://singularityhub.com/2017/06/21/is-there-a-multidimensional-mathematical-world-hidden-in-the-brains-computation/ +
      +

      5.3 Mapping entity relations in hypercube

      +
      +

      +Consider we want to create database of: +

        -
      • From there comes following idea: Maybe every problem can be -translated to geometry (use any shapes and as many dimensions as -you need). Solution(s) to such problems would then appear as -relatively simple search/comparison/lookup results. As a bonus, -such geometrical *data storage* AND *computation* can be -naturally made in *parallel* and *distributed*. That's what -neurons in the brain appear to be doing ! :) . Learning means -building/updating the model (the hard part). Question answering -is making (relatively simple) lookups (geometrical queries) -against the model. -
        • +
      • Books.
        • +
      • Authors.
        • +
      • Effort: Amount of time contributed by every author to every book +that he/she wrote.
      -
      • -
    -
    • + +

    +Information above can be represented as 3D cube where dimensions are: +

    +
      +
    • X: Book
      • +
    • Y: Author
      • +
    • Z: Effort
    + +

    +Points in that cube would nicely capture many to many relations +between authors and the books. +

    - -
    -

    2 Current status

    -
    +
    +

    6 Current status

    +
      -
    • More or less defined Vision / goal. -
      • +
    • More or less defined Vision / goal.
      • + +
    • Collected some inspiring ideas.
    • Implemented very simple persistent key-value map.
      • Long term goal is to use it as a backing storage engine and -implement more advanced features on top of this. -
        • -
      -
      • +implement more advanced features on top of this via layered +architecture. +
    -
    -

    3 TODO

    -
    +
    +

    7 See also

    +
    +

    +Interesting or competing projects with good ideas: +

    +
      -
    • check out Magma +
    • flexible user interface building for interacting with different data
        -
      • http://wiki.squeak.org/squeak/2665 -
        • +
      • Glamorous Toolkit +
          +
        • Moldable development environment. It is a live notebook. It is a +flexible search interface. It is a fancy code editor. It is a +software analysis platform. It is a data visualization engine. All +in one.
          • +
        • +
    - +
    + +
    +

    7.1 Computation on multi dimensional data

    +
    +
    -
    -
    -
    -

    Author: Svjatoslav Agejenko

    -

    Created: 2017-07-02 Sun 22:49

    -

    Emacs 25.1.1 (Org-mode 8.2.10)

    -
    +
    +
    +
    +

    Author: Svjatoslav Agejenko

    +

    Created: 2021-04-09 Fri 16:55

    +

    Validate

    +