Create block layout

[kdahlquist]

The first priority for @yshin4 after the release (#557) will be to work on creating a block layout. These tasks will need to be part of this (not necessarily in this order).

-Investigate what D3v4 already has "off the shelf"
-The nodes need to be laid out in a grid pattern (columns and rows), equally spaced to fill the area of the viewport.
-Once laid out, the nodes will not float or move like with the force graph layout.
-The nodes should be laid out in alphabetical order, left to right first, then top to bottom.
-We will need a menu item for toggling between the force graph and block layouts. Toggling back and forth should trigger a graph reload.
-Nodes can be moved in the normal way afterward.

More details will need to be worked out later, of course.

[kdahlquist]

@yshin4 needs to investigate what D3 already has for "grid" or "block" layout to see if code can be reused. cytoscape.js also has grid layout.

@kdahlquist suggests that @yshin4 should come up with an algorithm for how she thinks it should work so that she kind of knows what she is looking for.

@eileenchoe suggested a "snap to grid" feature, which is an old, old issue: #44

[yshin4]

Just some notes for me:

https://www.npmjs.com/package/d3-grid-layout
circle demo: http://hughsk.io/d3-grid-layout/
square demo: http://bl.ocks.org/herrstucki/5684816
force: http://www.puzzlr.org/force-graphs-with-d3/

[yshin4]

Moving off from Cytoscape because the demo seemed like it simply used their library instead of applying the algorithm within the source code I found for the grid-layout demo. I think if I were to redraw the graph using Cytoscape library, I would have to re-make the d3 force functions with Cytoscape.

But I found another demo made with d3 and I am trying to play around with the code right now.

[yshin4]

Tried to change the point array in the point grid layout to nodes or something else to see if it would work on various elements. Having trouble with append("g") with an error.. planning to get back to it.

[yshin4]

Current progress. Using d3-grid.js to produce a grid layout using rectangles. It has an easy way to implement one with a point as well, but not sure if "g" or the "node" for our grnsight will work as well. I spent my time understanding the d3-grid file itself, will attempt to apply with nodes soon.

[yshin4]

d3-v4 of the grid, I'm glad they have a v4 of it.
https://www.npmjs.com/package/d3-v4-grid
They also have node included for this one, will be trying to manipulate a simple one using fake data.

[dondi]

The nomenclature doesn't matter as much as how D3 expects to be given the data. The example above looks like the grid layout accepts an array of objects, which lines up with how GRNsight stores nodes. So a quick adaptation of the sample code would just replace the data argument with our node array.

[yshin4]

Made a new folder on my laptop to test out d3-v4-grid sample code, so I don't have to deal with all the grnsight stuff and instead focus on learning how v4 grid works. However, in order to use d3-v4-grid I need to import it. import Grid from 'd3-v4-grid'; , which html does not support. For such reason I could not try the sample code out and with the help and advice from @dondi, I will be looking into webpack and webpack tutorials so that I can actually try the code out.

[yshin4]

Fixed the import problem with webpack and attempted to use d3-v4-grid code. The code seemed to work without an error but could not make it to display the node. Tried different codes to fix this so I can see a visualized grid layout by exploring d3-v3 grid and https://bl.ocks.org/mbostock/4062045 this to make a small mock graph. But I still cannot figure out how the grid takes in data and displays it. Will be getting back to it next week.

[kdahlquist]

@eileenchoe suggests to pretend that the grid is a result of the manual node dragging. Go straight to GRNsight instead of trying to do it on sample code.

[yshin4]

two things to try:

1. Directly use the sample code on grnsight code (would make a branch for this) and see if the grid can work since it already has the visualization functions. Get the x,y position from the grid.node() and implement manual drag just like dragstart function. Make a button to trigger the grid layout.
2. Sample code visualization: node raw data => node array => grid

[yshin4]

I made a button, installed d3-v4-grid and webpack for my new branch. But I am having a lot of trouble with implementing the sample code directly on graph.js because once I select graph.js as the entry point for webpack, it does not know how to connect with other files such as upload.js. This means that files cannot communicate with which other, thus not being able to call global functions from other files. According to https://webpack.js.org/concepts/entry-points/, I can have array of strings for the entry point input so I did this:

but it still failed to communicate within the files and upload.js could not find sliderObject from sliders.js

@dondi is there a way I can set the a directory as an entry point (instead of a specific file) so that the files in the folder can all be bundled up together in bundle.js?

[dondi]

@yshin4 I think there is a misunderstanding here on what an entry point is. An entry point is like a “main” file—it is the top level of some cohesive body of code. Another way to think about an entry point is that it is the file that no other file depends on.

The idea of multiple entry points is to support the building of multiple “main”s —for example, if you have a web app consisting of multiple pages, and each page needs its own JavaScript file. We don’t have that in GRNsight. You really do have one entry point.

The way to solve your reference errors is to import/export your code. Right now we have upload.js and graph.js. What functions/objects/definitions are "exported" by each file? What functions/objects/definitions are needed by one file, which come from another file? The former becomes the exports. The latter becomes the imports or the requires (depending on what module approach you are going for—there’s the require style and the ES6 import/export style…you might need to read up on those).

In the end, I suggest that you think of a third file as your entry point: let's call it grnsight.js. This is the file that you list in the web pack config. From grnsight.js, you (a) import/require the top-level functions that you need, then (b) call them.

Welcome to life with true modules—something that JavaScript has not had in a practical sense until recently. However it does require some additional learning.

You might want to familiarize yourself with the whole module concept first; it's sort of the foundational assumption that webpack makes. See if this helps, and if things are still stuck we can address this on Monday.

[yshin4]

@dondi Ohhhh, no wonder I only found things like entry { page1: ..., page2: ..., page3:... } when I googled webpack stuff for multiple entries. That makes more sense now, thank you dondi.

I tried import/export earlier and had a slight hope that we can import and export the whole file and access it easily... I considered importing and exporting func/obj/def needed for other files but I wanted to make sure that webpack didn't have an easier way, since it is a lot of work to go through all the files and see which ones call what. I will be working on import/exporting (thankfully I am familiar with them from doing your classwork) for my next task now that I know!

And sorry but I am still confused with the grnsight.js part though. I see that a file is necessary for the webpack to let us use import (or require) and export. But how do we connect the functions together? For example, if 1) graph.js has a function called drawGraph and 2) upload.js calls drawGraph in the function displayNetwork, which 3) is triggered (called) when a button is pressed, how do we connect the two functions (drawGraph and displayNetwork) in grnsight.js?

[dondi]

For the last example particularly, you will then want:

1. drawGraph is part of what graph.js exports
2. upload.js imports drawGraph from graph.js so that it can use it
3. The role of grnsight.js is distinct here. It's more like, "what code gets everything started?" That is what I would expect to see in grnsight.js

So remember that the "entry point" is the actual script that the web page finally loads. So whatever gets executed immediately in that script is what I would expect to go in grnsight.js.

For example, if each file included an init as part of its exports that captures what needs to be done when the web page initially loads, that would be the call I would expect in grnsight.js. It would look something like:

import graph from './graph'
import upload from './upload'

graph.init()
upload.init()

The current structure of the GRNsight code does make it hard to suss this out, but it is in fact a good exercise to truly understand what is private to a file and what it exports, and finally...what it executes initially at page load. This latter part is what I would expect to be consolidated into an init function that the top-level grnsight.js would then call.

[yshin4]

Thank you dondi! I didn't have a init function but I was able to connect the files using grnsight as an entry point.

This is a note for myself in case I need to look back on how I imported/exported:
Webpack does not do module.exports, just export
https://github.com/webpack/webpack/releases/tag/v2.2.0-rc.5
Basic import and export
http://2ality.com/2014/09/es6-modules-final.html

[yshin4]

Drag documentation:
https://github.com/d3/d3-drag/blob/master/src/drag.js

For future:
https://github.com/d3/d3-hierarchy

[yshin4]

Able to organize the nodes in a grid, yay!

Next steps:

• Making the grid size the same as the viewport size.
• Alphabetizing the nodes.
• Padding the grid or shifting them so they are centered.

[yshin4]

Grid size changed to viewport size.
Currently working on centering the graph when layout applied.
Will implement undo (new button?) to reload the graph (or release fx,fy values).

[kdahlquist]

Looking good! Please make sure that there is some white space between the nodes and the edges of the viewport.

[kdahlquist]

In terms of the button, are you talking about toggling between block and force graph layout?

[yshin4]

Yes, will do. And yes I am talking about the toggling.

[yshin4]

Working on centering the graph when the graph layout is applied by calculating the distance between the leftMost/bottomMost Nodes and subtracting them from the container Width/Height. But the width margin does not seem quite right, investigating why this is so.

[yshin4]

TODO: make test files with only longest named nodes and see if the padding (spacing between the nodes) are appropriate.
TODO: check max number of nodes and number of edges to see if they all fit properly on the container.
TODO: make a undo button to free the fx, fy (forced? fixed?) by setting them all to null.

[yshin4]

Working on sorting the nodes in alphabetical order.

For the most part nodes are sorted properly as below.

But if the numeric literals are used as numbers instead of digits as below, E10 comes before E2 because the sorting method compares character by character and since 1 is less than 2, it puts E10 before E2.

I think this is acceptable because there is no way to check if the numeric literal in the node is a number or a digit. In fact, computers sort the files like this too (comparing char by char). And if the user wants to include numbers in the gene name, they can use A01, A02, ..., A10 to sort them properly.

[yshin4]

Implemented undo function when grid layout button is pressed again (toggle). I can also make a force graph button and block layout button instead of having the user toggle with one button if preferred.

Made a test-file test-files/edge-tests/thick_arrowhead_longname_weighted.xlsx in order to check for the longest and thickest node/edge case. Not everything fits in small viewport, as shown below.

We can go two ways:

1. either warn the users that they cannot load files that do not fit on small viewport and recommend larger viewport
2. or automatically change the viewport size to medium and load the graph.
   I like warning the users because I would prefer not to change graph settings without user's permission. We can further discuss this during the meeting.

[dondi]

When restricting graph to viewport, it's OK to let nodes overlap; this will be immediately visible to the user so they can decide on whether this needs to be dealt with.

Doing things automatically can backfire because the user might not realize that things like this are happening.

[yshin4]

Awesome, I will be implementing this to beta then.

[kdahlquist]

Reload button should use grid layout or force graph depending on what the user last used (i.e., don't change it back to force graph if a user had a grid, upon reload).

[yshin4]

@kdahlquist merged to beta, feel free to play with it. Meanwhile, we are working on the poster and presentation for 402 this week.

[kdahlquist]

The Grid Layout button should change its label to Force Graph Layout when the grid layout is enabled. I.e., the label should toggle back and forth depending on which layout is applied so the user can click it to change it to the other one.

The way it works right now is that you can't select Grid Layout before loading the graph. It seems like the user might want to be able to do this.

I think we need to

• add the grid layout to the client-side testing document
• work on the state controller Consolidate state management into a single controller for user selected options #576

Right now block layout is a little buggy when I click back and forth between the different viewport size options. It would be good to be able to generate a document that would systematically test this.

[yshin4]

New task according to feedback:

• manually reheat to unfreeze the graph between the toggles.
  @eileenchoe said to look for the .reheat method in sliders.js or graph.js
  The method was sliderController.simulation.alpha(1); but this did not fix my issue.
  I figured out how to recreate the issue, 1. load a graph 2. grid layout 3. click somewhere in the graph, i think it is okay as long as you click anywhere else other than the button itself? 4. go to another app, i used atom, and did an activity (like writing on and saving the file) 5. Click the button again, it does not work.

• Make the text on the button toggle back and forth according to the state (grid / force).

• Reset the text to Grid Layout once a new graph is loaded or when a graph is reloaded.

• Generate a script.

[yshin4]

New things up on the schedule:

• According to @kdahlquist when the sliders are locked, clicking the grid layout button does not change the layout to grid. Will investigate, I wonder if this is related to graph freezing between toggles.
• change the layout tab of the menu according to feedback
• make a PR from d3v4-grid to beta
• Consolidate state management into a single controller for user selected options #576

[yshin4]

Moving the remaining errors into a new issue.

[kdahlquist]

Live on v3.0.0.