Contexter in the browser

[billiegoose]

Hey Eduardo, I'm making a kinda cool File Navigator / Git-In-The-Browser thingy.

It uses BrowserFS to provide the Node.js fs API in the browser, and I'm working on adding support for fs.watch so I can do live-reloading and stuff.

Anyway, I don't have any concrete idea, but as I started thinking, "OK, I'll need a meta-data structure to track each folder's [open|closed] toggle state. And a meta-data structure to track each file's git [unchanged|modified|added|removed] state. And a meta-data structure to display a loader or spinner next to files while they're compiling..." and I remembered your project and thought I'd ping you because you have experience with that stuff!

[elingerojo]

Leaving aside "the big picture" for a moment, as I understand, you are looking for a UI that will "bubble" up the status of a file thru all its parent folders ...like the UI used at contexter-cli (...but with better styling ha ha).

Something like the color of the folders/files that have not been saved in ATOM.
Am I correct?

[billiegoose]

Exactly! The contexter-cli is what reminded me. (Side note: The File Navigator is a pure React component! I plan to release it as an npm package once it actually does all the things.)

Right now it uses two data structures, which I called data and statedata. The data is a JSON tree that mimics the folder structure:

It seemed logical at the time, and easy for users to understand probably. This would only change when files are added, removed, renamed.

And then I needed a way to toggle folders opened and closed and add "busy" indicators. Rather than mess with my existing structure, I decided to put "temporary state" in a separate structure I called statedata. But I was lazy and didn't want to translate "nde/FileNavigator/FileTreeView" into {nde: {FileNavigator: {FileTreeView: {}}}} so I thought I'd just use full paths like {"nde/FileNavigator/FileTreeView": {open: true}}.

But now I feel like its become needlessly complicated. 😞 I hadn't even thought about the "bubbling up" feature yet... how did you approach it in contexter-cli?

[elingerojo]

The approach really happens in contexter (not in contexer-cli)
There is only one data structure, an equivalent to your mentioned "statedata".
The data (or "file") is just one property in the structure.
As a matter of fact, the real data is rarely present in the data structure. Only metadata of the file is present (like path, filename, size, ...etc.)

The approach is to have only one metadata structure and render it as you like.
Your mentioned data structure could be just a render view of the metadata where you filter by path and filename

Having two different structures reflecting the same hierarchy is an open door for problems.

---

Maybe something like this:

{
  "fsEntity": {
    "fsEntity": {
      "name": "FileNavigator.js",
      "meta_1": "foo level 3",
      "meta_2": {
        "meta_2_1": "bar level 3",
        "meta_2_2": "baz level 3"
      }
    },
    "name": "FileNavigator",
    "meta_1": "foo level 2",
    "meta_2": {
      "meta_2_1": "bar level 2",
      "meta_2_2": "baz level 2"
    }
  },
  "name": "nde",
  "meta_1": "foo level 1",
  "meta_2": {
    "meta_2_1": "bar level 1",
    "meta_2_2": "baz level 1"
  }
}

Note: name is the folder name unless there is no "child" fsEntity. In that case, name becomes the file name because you reach the a "branch's leaf"

[billiegoose]

Having two different structures reflecting the same hierarchy is an open door for problems.

Exactly. I appreciate hearing how you structured it. My actual file tree code is... still quite messy. But meanwhile I've been working on git and gotten some more experience with representing file directory structures. Git actually uses two different structures! Trees, which represent a single directory, and the "index" which represents a snapshot of the entire repo. The tree form consist of multiple objects, linked in a directed graph (a tree). The git index consists of a single array of objects kept in sorted alphabetical order by full path name. Operations like "git add" and "git status" use the index, while the "git commit" operation converts the index into the tree form.

I think what I'll do is structure it more like the git index, where the "source of truth" is a Map<fullpath, metadata>. Then from that, derive the tree structure. This way, most operations (like rename, move) on individual files will be a quick and easy O(n) change to the metadata structure. Operations on directories will be much harder (moving a large directory will require renaming every file) but also less error prone (I can't accidentally put a file in two directories).

[elingerojo]

Wow!
I took a look at esgit and it is incredible.
Nice work.

---

Git actually uses two different structures

About the structures you referenced...
Both are very "linear" because both are array based. The holding structures for Trees is an array so it end up being an array of arrays.

I agree with you. For the "source of truth", a Map<fullpath, metadata> makes a lot of sense 'cause it is "similar/compatible" with git index structure. So parsing, extracting, extending, rendering operation between both will be easy.

In the other hand, the React UI will be a "real" Tree structure where you will be able to move any size directory in a breeze (...in the UI only).

A couple of functions to go from one structure to the other and viceversa will do the trick.

The way I see it is: The fullpath is both a <big key> and a<sub-keys list> at the same time.

When working in the Map, it is a <big key> (...or a concatenated <sub-keys list>) .
When working in the UI it is a <sub-keys list> (...or a split <big key>).

The premise is that both are the same all the time (if one changes, for whatever reason, the other must reflect it).

Point 1
This leads to challenge the assumption that the best structure is the Map because the React UI Tree is interchangeable with the Map. So both end up being "the same". Both are good choices.

The frequency of operation will favor the React UI Tree assuming that the user operations are more UI related like creating, modifying, moving, deleting files than the git operations sparsely used during a session.

Point 2
But from the Nde developing point of view, the logical next step is the Map not the UI. This will make a robust code base. Maybe, just keeping an eye on the UI structure for future in case there are performance issues.

Point 3
Both could be the "source of truth" ...but at different time/step.
If a "perfect" translation between React UI Tree and Map structures could be coded, then the "source of truth" could be interchanged depending on the operation performed. When doing git operation, the Map. When doing UI operations, the React UI Tree

Point 4
Skipping the Map all together. Going from git index and the array of trees directly to the React UI tree. I don't know how practical could that be. Need to learn more about Nde

Just forked it to understand better.

[billiegoose]

Need to learn more about Nde

Just forked it to understand better.

Oh no. What have I done?! Um... it's a mess. You can play with the live demo though: https://nde.now.sh
(it takes a while to load, but caches everything so it loads much faster the 2nd time. It should work offline, although that seems slightly broken.)

Edit: Oh, it does work offline. I just had both "Offline" and "Bypass for network" checked in Chrome's devtools. Those two settings don't mix well.

[elingerojo]

No problem. It works OK.
I know it is version 0.0.0 and a bumpy ride is expected.

[billiegoose]

Haha, awesome you got it to work. I got git().push() working this past week so I'll be adding UI elements for staging, committing, and pushing this week!