Draft: Data Flow / Graph based implementation of PySwitch

[Tunetown]

PySwitch suffers from some limitations:

• RAM on the MC devices
• Flexibility: Functionality from one callback/action cannot be easily adapted to others when the base classes differ etc.

NEW IDEA: Rewrite completely based on data flow (topologically sorted DAGs). Provide more atomic/minimal building blocks instead, and provide high level actions by reusable "Sub-Patches", which are flattened when run on the device. Think of the RISC idea, or Unix Command Lines: Provide simple modules to build more complex algorithms, instead of a monolithic framework like PySwitch is at the moment. The end goal would be:

• Probably huge RAM savings:
  • Because most of the code would be gone, the logic is stored in a node list only which is minimal in RAM, also processing could be done way more flat in terms of stack size which has also been a pain in the ass sometimes during development. The stack size will basically be totally flat all the time.
  • Avoid object orientation. Object oriented programming is superior in terms of software development in general (no discussion for me), but costs lot of RAM, and this implementation would by logically modular by design, and you could use OO in modules by wish anyway. After all, the non-OO approach might be better suited for this kind of embedded device.
• Complete flexibility! This could be used for anything as blocks could be user defined.... you know the Nord Modular patching screens? Yes, drawing cables in a big patchbay....
• Still possible to keep the Emulator UI similar to the current one by providing blocks. It will be different, but an idea is to allow special connection types to bundle stuff so the non-technical user can just click his stuff together like now, but the more technical people could dig in deeper easily and graphically in the same platform.
• Better suited for Python beginners as there is no OO knowledge needed to write a simple block which does something. The interface will be pretty simple, so writing you own modules can be a beginners exercise.

DRAWBACKS:

• Complete incompatibility: No old configs can be taken over per se. However, all actions will be rewritten as blocks 1:1, so there could be a simple converter option in the (old) emulator to convert to the new graph JSON format. I think about that when the basic engine is OK.

Outline:
Blocks:

• Building blocks should be interconnectable on the base of connection types similar to Max or Pure Data. There could be these types of interconnects:
  • MIDI messages
  • Internal messages (like the "bang" in Max/PD, also MIDI and drawing commands)
  • Boolean States
  • Numbers
  • Strings
• Replace all actions and callbacks with pre-made "circuits" the user can choose from (or build his own!)
• Instead of Inputs and Displays lists, just provide the IO (switches, labels etc., also MIDI over USB/DIN) as "inlets" and "outlets" like Max/PD
  • LEDs with individual addressing, putting the now-existing distribution algorithm in a module ;)
  • Switches including all modes (momentary, latch etc), sending messages or providing boolean state outlets, too
  • Display Labels (Inputs: Text, Color)
  • Encoders/Wheel etc.
• Provide a graphic "breadboard" for editing actions, similar to Max/PD
• Remove communication.py (MIDI routing can then be done freely using interconnects) and config.py, you now have just a patch.json file
• Store as Net list
• Provide elementary building blocks, with defined/typed inlets and outlets, implemented in Python and extensible.
  • All kinds of triggers (MIDI -> bang, bool -> bang, number -> bang etc.)
  • Kemper Bidirectional Protocol as module
  • All kinds of logic blocks etc. (provide a basic library of simple objects)
• Remove UI code, directly address hardware in the IO blocks
• Implement better MIDI drivers

Graph Processing (full processing, done at the start only):

• The data flow is a DAG (Directed acyclic graph), so we can use topological sorting. The engine has to create and sort the DAG on startup only, after that only the parts which have changed are recalculated, only needing index list access and bit masking etc., so this can be fast!
• raise error in case of cycles in the graph

Emulator

• Use the PySwitch / Pyodide environment as it is running stable and suits well
• Use JointJS to render the graph visually, inkl. editing
• Keep Tab Subsystem for:
  • JSON Patch editor
  • MIDI Monitor
  • Virtual devices
  • Editors for selected node
    • Properties
    • Sources
  • Install/Update:
    • Just open the MC drive via Web File API (already used in the emulator to open local folders)
    • Manage the installation and updates with one click! Scan what is there, update/add libraries, scan and update code.py and boot.py (should be pretty fast because reading the drive is fast, just writing is slow as hell. Only the delta is updated.)
    • Dont use the MIDI bridge. Using Web File API is superior in any perspective.

[Tunetown]

Update for the ones who are interested:

I got the basic engine working like a charm! Display drawing (inkl. Text), MIDI, LED drivers, Switches, its all there already, and runs with remarkable performance:

• Basic engine only consists of about 600 lines, one file, and takes only 10k of RAM with a blank patch (!!). That leaves 95% of RAM for the patch (which will take some because the functionality must be somewhere, but the data flow engine is VERY effective by design, thats why i chose to go that way. Reality proves this already in the prototype.
• With all all hardware assigned (display, MIDI, LEDs etc, just the drivers) we are still at merely 25k of usage. Thats because i did the following:
• Write my own display drivers, directly talking to the frame buffer memory. This means: Any amount of labels with NO extra RAM usage! I had 50000 rectangles drawn and still not a kilobyte more usage. Sure, the driver is basic, and really not optimized for animations for example, but this is not the point on those devices anyway. If one wants to animate, she/he can write own modules using the adafruit drivers, there will be documentation on that later.
• Write my own MIDI drivers, directly talking to UART/USB midi. This does not only take way less RAM than the original adafruit drivers we used in PySwitch, but also allows for arbitrary length messages. The original drivers are by design not working well with larger Sysex, but for the new driver i did stop testing at around 10000 (!) MIDI half bytes in one sysex message transferred successfully without any issues via USB or DIN (no matter), after that it was fine for me :D
• The UI will be totally slick. I will post previews but i really have fun with it already ;D

Stay tuned.

[at77101]

Nice work, @Tunetown! Excited for the update!

[seve54]

Thank you so much for the great work you're doing! We're all waiting for you!

[AppleTreeChen]

Thank you very much for your efforts. We look forward to its official launch.

[seve54]

Having the MC10 wireless version (with Midi Mate), it would be interesting to be able to use it with the new version! Thanks for the great work.

[chrjohns]

Checking in on this. Following with interest. How’s it going?

[Tunetown]

Short update: I had not much time in the last weeks, and was on vacation. There are still basic things to do and as i said its a bigger project, but i will continue as soon i can :) Good to know people are interested, this will keep me going for sure ;)