JamberryPi
Peter Friend, Ian Vermeulen, Haoyuan Chen
Objective
The objective of our project is to construct a multi-user music track mixer using the Raspberry Pi and an external speaker. Users will be able to generate and edit their own sound tracks and collaboratively create mixed music on the server running on the Raspberry Pi. Sound tracks are sent from the front end web UI to the back end server and played in a loop.
Introduction
Inspired by open source live-coding and audio collaboration, using the Raspberry Pi, we have designed and built a sound synthesizer system. In this system, there is a front end web UI that users will be able to log onto and manage their music tracks. Upon user submission, the sound tracks will be sent through http to the back end server that process the track information and play out the mixed music by looping all tracks it received from all users. In principle, the system is able to support an infinite number of users and we didn’t encounter bottlenecks when multiple users are using the system simultaneously.
This project was made possible through use of online resources and libraries, which will be further discussed in the design section. Since this project involves the use of multiple languages and multiple libraries, we spent a lot of time exploring the correct way to implement and integrate each part of the system. It was not a straightforward implementation process but finally we successfully achieved our goal in building a working multi-user collaborating sound synthesizing platform.
Design
The basic design idea is to have a front end web UI that enables users to manage their music tracks and a back end server that keeps the states of submitted tracks and play out all the mixed tracks recursively. The front end UI is composed of HTML pages mainly written in javascript and the back end server involves the usage of a collection of open source software for server side web development in Clojure language. The communication between the front end and back end server is realized through RESTful over http.
Front end
The front end consists of simple HTML pages. The initial page is a type of login page through which the user accesses the track management. During login the user is given an ID by the server that is used to track the session. The ID given is also used to make sure each user only have access to their own music tracks to ensure system security.
The main page is dynamically populated with elements via Javascript. Tracks are added and removed as needed. User could add new tracks by first selecting an instrument from the drop down list on the page and then click on the “Add a track” button. A track consist of a bunch of checkbox elements that users can check to designate wanted notes and beats. Each new track is assigned a track-ID by the server upon submission. This track-ID is then sent with any future requests to the server in order to modify or remove it. Modification of previously submitted tracks is done when a user hit the submission button again. Submitted tracks can also be removed. Once the remove button is clicked, the corresponding track will be removed from the html page and a remove request with the track’s ID will be sent to the server. The server will then discard that track and not play it in the next cycle of music playout.
Communication with the server is done using AJAX. This allows us to asynchronously communicate with the back end without reloading the page. We send data simply by using form data and a POST request.
Back end
The backend is divided into five logical components. We wrote the webserver and the tracks/synth abstraction in Clojure. Overtone is an open source Clojure library providing clojure bindings for Supercollider. Supercollider and Jack are both native-code daemons we start as background tasks from a shell script. IPC to Supercollider is via Linux sockets and the connection from Supercollider to Jack is abstracted away from our sight. Interestingly, Overtone can connect to Supercollider over the network, so our application could be further split across multiple computers.
Jack and Supercollider
The Jack and Supercollider processes are long running daemons which need little caretaking. Once jack is started, we scripted “connecting” Supercollider to Jack (Jack intends us to imagine physical patchbay wires form the analog synth days of yore), so Supercollider depends on Jack and exits if Jack exits. However, different Overtone sessions (different runs of our Clojure program) can come and go from the single active Supercollider process without issue. While both Jack and Supercollider are cross platform, our script contains platform specific options.
Overtone
Overtone is intended primarily for “live” use inside an interactive programming environment (“REPL” in clojure terminology), thus our usage stretches a little beyond what it is primarily for. In particular, we use the definst macro, which can only be compiled when Overtone is connected to a Supercollider server. The result is that we defer compilation of a tiny part of the program by “quoting” it (a Lisp concept) and evaling it (compile and run) at a carefully controlled time in our system startup. This requires careful program design because we cannot simply assemble a namespace at compile time, we need to store in a variable populated at runtime references to the instruments produced by definst.
Tracks and Synth
Tracks bridges the gap between tracks as understood by the UI and (which tone and note-subdivision checkboxes are checked) and an internal track data structure. The internal structure is ordered and stores frequencies rather than note names. This can be seen in params-to-track in track.clj
We use a technique called “temporal recursion” to repeatedly play tracks. In temporal recursion a “play all the tracks” function recurses through time (rather than stack). This function schedules all the notes in all the tracks to play at appropriate times and then schedules itself to run again when it is time for the next loop. This can be seen in loop-tracks in synth.clj.
Webserver
The webserver inhabits the Clojure Ring ecosystem, a loose collection of open source software for server side web development in Clojure. Ring provides abstractions for http requests and responses, http-kit is a general purpose http server, Compojure provides “routing” (invoking appropriate handlers when certain URLs are loaded), and Buddy provides authentication and authorization. The webserver both serves static resources such as UI webpages and client-side scripts (found in jamberrypi/resources/static) and API endpoints (defined in my-routes in webserver.clj). The API endpoints provide basic operations like creating a user, adding a track, updating a track, and removing a track. The track operations simply call through to the Track namesapce.
We keep track of users using Buddy authentication. In our configuration Buddy uses Ring Sessions (which places a unique session id in an HTTP cookie) to keep track of users, but Buddy could be reconfigured to use an alternate mechanism such as tokens which must be sent back and forth in every request. Buddy authorization, which we use only minimally, controls which users can access which URLs. Because no attempt was made to harden this application, authorization and authentication don’t do too much besides ensuring that users take particular paths through our application UI and APIs.
Testing
Supercollider ran effortlessly on the Pi because it was already compiled and in the repositories (in fact, preinstalled on Raspbian). Similarly, Jack 2 (the dbus edition) came compiled and working on our stock Raspbian already. However, Jack did not quite work out of the box. While sparsely documented, to output something even resembling clean audio on the Pi’s build in audio card it is necessary to increase the playback latency from the default of two “periods” to three.
Luckily it was not necessary to bridge Jack to Alsa by sampling from Jack in software and playing back into Alsa, Jack acquired the audio card through its confusingly named Alsa backend without issue.
Getting Overtone working required a tricky hack (source in the references section) that involved replacing a broken dependency with a different version.
Initial backend testing assumed that it would be possible to Ahead of Time compile all of our Clojure code (Clojure can normally be compiled into JVM bytecode for performance and faster application load times). After extensive experimentation, and eventually reading Overtone source code, it became apparent that the definst macro contains a compile-time side effect which requires Overtone to be connected to Supercollider. Because of the way namespaces (the compilation unit of Clojure) are transitively loaded as dependencies, it was very hard to both defer compilation of code using definst and to make variables and functions available in the right namespaces. The working design uses bad style and can be seen in the init function of synth.clj.
Some of the UI could be tested in isolation (i.e. when loaded from a file on a computer without the webserver), but many parts could only be tested with a working webserver. Due to the short development cycle we did not make a mockup server, we simply tested which as complete a backend as was available at any given point in time.
Because much of this system is focused on user interaction, integration testing took the form of using the system from a web browser many times.
Drawings
Figure 1. Front End UI Welcome Page
Figure 2. Front End UI Main Page for Track Edit
Figure 3. Front End UI Main Page with Tracks Added
Figure 4. Front End UI Main Page with Tracks Edited
Results and Conclusions
Everything performed as we planned and we successfully achieved our initial goals. Multiple users are able to log onto the web UI and collaboratively generate mixed music by managing their sound tracks. Sound tracks can be added, modified, updated and deleted in real time accurately. The server never crashed and no noticeable mistakes in the played out music were found during testing and demo. The whole system is able to run smoothly and flawlessly. However, the sound quality is not very high, but since sound quality was not a main focus of our project, it would not be of concern. There are still potential in this project and improvements in many aspects could be made if we are allowed more time.
Current Difficulties and Future Work
Note Compression
For instruments that have tones, when multiple adjacent subdivisions have the same tone played by the same instrument (several checkboxes next to each other aligned horizontally), they can be combined into one single note (one entry in the data structure instead of several). The infrastructure already supports notes of arbitrary length, thus all that is necessary is a preprocessing pass on the data structure. This would improve sound quality by making sustained notes. The length of notes is audible because there is a slight shaping of the onset and falloff of each note.
Security
Authorization could be relatively easily made more restrictive, only allowing users to edit their own tracks (as it is now, a motivated user could guess track ids for other users and remove tracks belonging to the other user). The beginnings of this can be seen in owns-track in webserver.clj. Https could also be used to prevent someone capturing packets on the network from impersonating another user, but it would require a TLS certificate, which in turn requires trusting our application with a valuable private key.
Front End UI Improvement
Now the front end UI is just an initial version, the graphics are not well designed and the user experience is not very well. Modifying sound tracks take quite some time as users have to manually click each checkbox in order to modify the track. This can be improved by adding features such as a check-all-box button that will check all checkboxes on a row or enabling mouse drag detection to select a group of boxes instead of only one box per mouse click.
Sound Quality Improvement
The music coming out of the speaker now is not of high quality. There is constant white noise in the background, even when there is no sound tracks playing. This could due to the hardware glitches inside the Raspberry Pi and finding a way to improve sound quality is not going to be an easy task since the Pi audio output was not designed to be very high .
Starting the Back End Server faster
Currently it takes a long time, around 2 minutes, to start the back end server. The two addressable sources of this problem are: compiling Clojure source code when loading the program and the build system requiring a second JVM.
While normally which Clojure namespaces are ahead-of-time compiled can be freely chosen to perform performance and usability tradeoffs, previously discussed Overtone problems make it unclear which parts of our system can be ahead-of-time compiled. At present nothing is, everything is compiled when the system starts, but ahead-of-time compiling some namespaces could make program start much faster.
Additionally, we have been running the system through the build tool Leiningen, which involves running one JVM for Leiningen and another for the main application. Each JVM takes a fair amount of memory and loads fairly slowly. This is orthogonal from the issue of when code is compiled, because Leiningen can generate a standalone executable jar that includes the Clojure compiler and compiles Clojure source code when it is loaded. Leiningen “uberjar” can generate an executable Jar that does not involve Leiningen when it runs. This should work with little effort but we have not tried it yet.
Contributions
Peter Friend Ian Vermeulen Haoyuan Chen
Back End Server Front End UI Front End UI
Front End UI Testing Integration
Integration Integration Debugging
Testing Debugging Testing
Debugging Webpage Webpage
Webpage
Materials
- RPi 2 Model B x 1 $35
- Speaker x 1 $11
- Total $46
References
- https://github.com/overtone/overtone/wiki/Getting-Started
- http://7-themes.com/7039581-hd-music-background.html
- http://www.w3schools.com/
- http://wiki.linuxaudio.org/wiki/raspberrypi and http://lists.linuxaudio.org/pipermail/linux-audio-user/2012-October/087650.html (Jack configuration)
- https://gist.github.com/rogerallen/82dc82c9079eef6bc456 (hacking Overtone to work on Pi)
Thanks
Special thanks to professor Joseph Skovira and teaching assistants Gautham Ponnu, Jacob George and Jingyao Ren.
Code Appendix
export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/dbus/system_bus_socket
startup.sh
# environment variable so we can find a dbus without xorg source ./prepare_jack.sh # start jack. -n3 is the magic for audio quality jackd -m -p32 -d alsa -n3 -r44100 & sleep 2 # supercollider on UDP port 4555 scsynth -u 4555 & sleep 2 # connect supercollider virtual audio cables to jack playback jack_connect SuperCollider:out_1 system:playback_1 jack_connect SuperCollider:out_2 system:playback_2
README
Requirements: Java 8 and the Leiningen build system. Leiningen must be on your path and set up by following the Leiningen install instructions. Starting Jack and Supercollider: From this directory, run startup.sh. Wait a bit (about 20 seconds) until things stop appearing in stdout. Jack ("jackd") and Supercollider ("scsynth") have been started in the background and will occasionally print to stdout/stderr. To kill both, kill jackd and the Supercollider process will follow. Starting our system: Change into the jamberrypi directory, then run `lein run`. Wait until the good to go message appears (after a bunch of Overtone junk is output).
project.clj
(defproject jamberrypi "0.1.0-SNAPSHOT" :description "Cooperative rhythm-jam synth game built on Overtone" :url "http://example.com/FIXME" :license {:name "Eclipse Public License" :url "http://www.eclipse.org/legal/epl-v10.html"} :dependencies [[org.clojure/clojure "1.8.0"] [overtone "0.9.1" :exclusions [clj-native]] [clj-native "0.9.5"] [org.flatland/ordered "1.5.3"] [compojure "1.5.0"] [ring "1.4.0"] [ring/ring-defaults "0.2.0"] [http-kit "2.1.18"] [ring/ring-json "0.4.0"] [buddy/buddy-auth "0.13.0"] [buddy/buddy-core "0.12.1"]] :main ^:skip-aot jamberrypi.core :target-path "target/%s" :profiles {:uberjar {:aot :all}})
tracks.css
h1 { color: white; text-align: center; } a:link, a:visited { background-color: #f44336; color: white; padding: 8px 15px; text-align: center; text-decoration: none; display: inline-block; } a:hover, a:active { background-color: red; } .btn { border-radius: 4px; border-width: 0px; background-color: white; margin: 5px; } .instrument-label { color: #FFFFFF; padding-left: 10px; } .notecell { font-size: 14px; background-color: #FFFFFF; color: black; width: 20px; padding-left: 5px; }; .even-beat { background-color:#9FFF9D; } .odd-beat { background-color:#fde99a; } body{ background: url("6919714-music-background.jpg"); background-size: 100% }
get-started.html
<!DOCTYPE html> <html> <head> <title>Welcome to jamberrypi!</title> </head> <body> <h1>Click the button to get started!</h1> <form method="POST" action="/create-user"> <input type="submit" value="get started" /> </form> </body> </html>
retrieve_instrument.js
/** retrieves instrument information using AJAX data back in the form "{"saw-note":...}" **/ function instrument_retrieval(instrument_map) { var url = "/list-instruments"; $.getJSON(url, function(data) { var select = document.getElementById("instrument_dropdown"); $.each(data, function(key,val) { instrument_map[key] = val; var option = document.createElement('option'); option.value = key; option.text = key; select.appendChild(option) }); /** $.ajax({ url : url, data : data, dataType : 'json', success : function(data) { console.log("success: retrieve-instrument"); } }); **/ }); }
track-manage.html
<DOCTYPE html /> <html> <head> <script type="text/javascript" src="./tracks.js"></script> <script src="jquery-2.2.3.min.js"></script> <!--<script type="text/javascript" src="submission.js"></script>--> <script type="text/javascript" src="retrieve_instrument.js"></script> <link rel="stylesheet" type="text/css" href="./tracks.css" /> <title>Edit tracks</title> </head> <body onload="init()"> <h1>Edit my tracks</h1> <select id="instrument_dropdown"> </select> <a href="#" onclick="add_track_wrap();" class="btn"> Add a track </a> <div id="track-container"> </div> </body> </html>
tracks.js
var instrument_map = {}; function init() { track_container = document.getElementById("track-container"); instrument_retrieval(instrument_map); num_divisions = 48; } function create_cell(html_row, note, division) { var cell = html_row.insertCell(); var checkbox = document.createElement("input"); checkbox.type = "checkbox"; checkbox.name = note + "," + division; checkbox.setAttribute("index", division); cell.appendChild(checkbox); return cell; } function create_colgroup(beat) { var fs = document.createElement("colgroup"); var beat_class = "beat" + beat; // mark even-odd beats for styling var alternation_class; if (beat % 2 == 0) { alternation_class = "even-beat"; } else { alternation_class = "odd-beat"; } // set class. Spaces separate multiple classes applying to same element fs.setAttribute("class", beat_class + " " + alternation_class); fs.setAttribute("span", 4); return fs; } function make_row(table, note_name) { var row = table.insertRow(); var note_cell = row.insertCell(); note_cell.className = "notecell"; note_cell.innerHTML = note_name; for (var division = 0; division < num_divisions; division++) { var cell = create_cell(row, note_name, division); } return row; } function add_track_wrap () { var select = document.getElementById("instrument_dropdown"); var selected = select.options[select.selectedIndex].value; var note_names = instrument_map[selected]; add_track(selected, note_names); } function add_track(instrument_name, note_names) { var num_rows = note_names.length; var track_div = document.createElement("div"); var instrument_dis = document.createElement("h3"); instrument_dis.innerHTML = instrument_name; instrument_dis.className = "instrument-label"; track_div.setAttribute("class", "track_div"); var instrument_field = document.createElement("input"); instrument_field.type = "hidden"; instrument_field.name = "instrument"; instrument_field.value = instrument_name; var submit_button = document.createElement("input"); submit_button.type = "submit"; submit_button.className = "btn"; var remove_button = document.createElement("input"); remove_button.type= "button"; remove_button.className = "btn"; remove_button.value = "Remove Track"; remove_button.onclick = function() { // questionable jquery alert var trackID = $(this).siblings('input[name=track-id]'); if (trackID.length > 0) { remove_track(trackID[0].value); } $(this).parents('.track_div').remove(); }; // make form var track_form = document.createElement("form"); track_form.onsubmit = function() { // this runs the first time we submit // extra level of function to get a javascript closure return function(e) { submit_button.disabled = true; console.log("button disabled"); var form_data = new FormData(track_form); var ajax = new XMLHttpRequest(); ajax.addEventListener("loadend", function(e) { submit_button.disabled = false; console.log("button enabled"); }); // this runs when we are done submitting (creating) ajax.addEventListener("load", function(e) { var track_id = JSON.parse(this.responseText)["track-id"]; // convert this form to an edit form var id_elem = document.createElement("input"); id_elem.type = "hidden"; id_elem.name = "track-id"; id_elem.value = track_id; track_form.appendChild(id_elem); track_form.onsubmit = function() { // this runs the second time we submit return function(e) { submit_button.disabled = true; console.log("button disabled"); var ajax = new XMLHttpRequest(); ajax.addEventListener("loadend", function() { submit_button.disabled = false; console.log("button enabled"); }); ajax.open("POST", "/update-track"); var form_data = new FormData(track_form); ajax.send(form_data); e.preventDefault(); }; }(); }); ajax.open("POST", "/add-track"); ajax.send(form_data); e.preventDefault(); }; }(); // make the colgroups (horizontally across, one for every 4 divisions) var table = document.createElement("table"); var note_names_colgroup = document.createElement("colgroup"); note_names_colgroup.setAttribute("span", 1); table.appendChild(note_names_colgroup); for (var i = 0; i < Math.floor(num_divisions / 4); i++) { table.appendChild(create_colgroup(i)); } // make the rows for (var i = 0; i < num_rows; i++) { make_row(table, note_names[i]); } // add everything track_form.appendChild(table); track_form.appendChild(instrument_field); track_form.appendChild(submit_button); track_form.appendChild(remove_button); track_div.appendChild(instrument_dis); track_div.appendChild(track_form); track_container.appendChild(track_div); } function remove_track(trackID) { var url = "/remove-track"; var data = { "track-id" : trackID }; console.log("removing id " + trackID); $.ajax({ type: 'POST', url: url, data: data, //dataType: 'json', success: function(data) { console.log("success: remove-track"); } }); }
webserver.clj
(ns jamberrypi.webserver (:require [jamberrypi.track :as track] [ring.middleware.defaults :as ring-defaults] [ring.middleware.resource] [ring.middleware.content-type] [ring.util.response :refer [response content-type redirect]] [org.httpkit.server] [buddy.auth :refer [authenticated?]] [buddy.core.nonce] [buddy.core.codecs] [buddy.auth.accessrules] [buddy.auth.middleware] [compojure.core :refer [defroutes GET POST ANY context]] [ring.middleware.json] [buddy.auth.backends])) (def ^:private current-user-id (atom -1)) (def ^:private user-tracks "Map from user id to collcetion of tracks owned by user." (atom {})) ;; =============================================== ;; Authentication ;; =============================================== (defn ^:private create-user "Create a new user and return the token for it." [] (let [user-id (swap! current-user-id inc)] (swap! user-tracks assoc user-id []) ;; store that this id has no tracks user-id)) ;; return id itself (defn ^:private login "Precondition: users are not already logged in." [] (let [newly-made-id (create-user)] (-> (redirect "/track-manage.html") (assoc-in [:session :identity] (str newly-made-id))))) ;; =============================================== ;; Authorization ;; =============================================== (defn ^:private on-access-error [request value] (-> (ring.util.response/response "Not authorized") (ring.util.response/status 403))) (defn ^:private signed-in [request] (:identity request)) (defn ^:private owns-track [{user-id :identity {track-id "track-id"} :params}] (contains? (@user-tracks user-id) track-id)) ;; documented in http://funcool.github.io/buddy-auth/latest/#access-rules (def ^:private access-rules [{:uri "/create-user" :handler #(not (authenticated? %))} {:uri "/add-track" :handler authenticated?} {:uri "/remove-track" :handler {:and [authenticated?]}} {:uri "/update-track" :handler {:and [authenticated?]}} {:uri "/track-manage.html" :handler authenticated?}]) ;; =============================================== ;; Routes ;; =============================================== (defroutes my-routes (GET "/" [] (redirect "/get-started.html")) (POST "/create-user" request (let [newly-made-id (create-user) new-session (-> (request :session) (assoc :identity newly-made-id))] (-> (redirect "/track-manage.html") (assoc :session new-session)))) (POST "/add-track" [instrument & params] (let [track (track/params-to-track instrument params) id (track/add-track track)] (-> (response {:track-id id}) (content-type "application/json")))) (POST "/remove-track" [track-id] (response (track/remove-track (Integer/parseInt track-id)))) (POST "/update-track" [track-id instrument & params] (let [track (track/params-to-track instrument params)] (track/replace-track (Integer/parseInt track-id) track) (response ""))) (GET "/list-instruments" [] (-> (response (track/instruments-and-notes)) (content-type "application/json")))) ;; =============================================== ;; Middleware ;; =============================================== (def ^:private wrapped-handler ;; modify site-defaults to my preferences. We turn off things that are ;; uneeded for this kind of embedded website. (let [my-ring-opts (-> ring-defaults/site-defaults (assoc-in [:security :anti-forgery] false) (assoc :session false)) access-rules-opts {:rules access-rules :on-error on-access-error} auth-backend (buddy.auth.backends/session)] (-> ;; wrap handler my-routes with my version of the defaults (ring-defaults/wrap-defaults my-routes my-ring-opts) ;; convert data-type bodies to json and application/json request to data (ring.middleware.json/wrap-json-body {:keywords? true}) ; request (ring.middleware.json/wrap-json-response) ; response ;; wrap handler with middleware that serves static pages out of resources (ring.middleware.resource/wrap-resource "static") ;; also send the right content type for those resources (ring.middleware.content-type/wrap-content-type) ;; authorization (buddy.auth.accessrules/wrap-access-rules access-rules-opts) ;; authentication via tokens, sets :identity (buddy.auth.middleware/wrap-authentication auth-backend) (ring.middleware.session/wrap-session)))) (defn start-webserver [] (println "Starting webserver on port 80") (org.httpkit.server/run-server wrapped-handler {:port 8080}))
(ns jamberrypi.synth (:use [overtone.core] [flatland.ordered.map :refer [ordered-map]])) (def insts (atom nil)) (def ^:private inst-has-tones {"saw-note" true "noise" false "blip-note" true "cub-note" true "par-note" true "kick" false "dubkick" false "snare" false}) (def note-names "Hardcoded map from note name to frequency, see http://www.phy.mtu.edu/~suits/notefreqs.html" (ordered-map "A4" 440.0 "G4" 392.0 "F4" 349.23 "E4" 329.63 "D4" 293.66 "C4" 261.63 "B3" 246.94 "A3" 220.00 "notone" :notone)) (def ^:private note-bpm 60) (def ^:private division-bpm (* 4 note-bpm)) (def ^:private division-nome (metronome division-bpm)) (defn init [] (eval '(do (use '[overtone.core]) (require '[overtone.inst.drum :as drum]) (definst saw-note [freq 440 length 1] (* 0.1 (env-gen (lin 0.1 length 0.1) 1 1 0 1 FREE) (saw freq))) (definst noise [length 1] (* 0.1 (env-gen (lin 0.1 length 0.1) 1 1 0 1 FREE) (white-noise))) (definst blip-note [freq 440 length 1] (* 0.1 (env-gen (lin 0.1 length 0.1) 1 1 0 1 FREE) (blip freq))) (definst cub-note [freq 440 length 1] (* 0.1 (env-gen (lin 0.1 length 0.1) 1 1 0 1 FREE) (lf-cub freq))) (definst par-note [freq 440 length 1] (* 0.1 (env-gen (lin 0.1 length 0.1) 1 1 0 1 FREE) (lf-par freq))) (reset! jamberrypi.synth/insts {"saw-note" saw-note "noise" noise "blip-note" blip-note "cub-note" cub-note "par-note" par-note "kick" drum/kick "dub-kick" drum/dub-kick "snare" drum/snare})))) (defn instruments-and-notes "Returns map associating each instrument name with seq of notes it supports" [] (let [true-notenames (remove #(= "notone" %) (keys note-names)) acc-notenames (fn [acc inst-name] (if (inst-has-tones inst-name) (assoc acc inst-name true-notenames) (assoc acc inst-name ["notone"])))] (reduce acc-notenames {} (keys @insts)))) (defn ^:private divs-to-seconds [divs] (* (/ divs division-bpm) 60)) (defn play-instrument [instrument freq len] (let [inst (@insts instrument)] (if (= freq :notone) (inst) (inst freq (divs-to-seconds len))))) (defn play-note-based-on "Schedule note to play at appropriate division, assuming base-time is division 0" [base-time instrument {tone :freq div :division len :length}] (let [target-time (division-nome (+ base-time div))] (apply-at target-time play-instrument instrument tone len []))) (defn play-track "Play a track once." [base-time {instrument :instrument notes :notes}] (let [note-at-time (fn [_ elem] (play-note-based-on base-time instrument elem))] (reduce note-at-time nil notes))) (defn loop-tracks "Loop forever, playing whatever tracks are available." [tracks-atom num-divisions] (let [base-time (division-nome) next-time (+ base-time num-divisions) each-track (fn [track] (play-track base-time track))] (apply println "looping ids: " (keys @tracks-atom)) (doseq [track (vals @tracks-atom)] (each-track track)) (apply-at (division-nome next-time) loop-tracks tracks-atom num-divisions [])))
(ns jamberrypi.core (:gen-class) (:require [jamberrypi.webserver] [jamberrypi.track] [overtone.core])) (defn -main "Startup the clojure part of the system (does not include jack or scsynth)" [& args] ;; fire up control webserver (jamberrypi.webserver/start-webserver) (overtone.core/connect-external-server 4555) (jamberrypi.track/play-all) (println "Startup complete, good to go!"))
(ns jamberrypi.track (:require [jamberrypi.synth :as synth])) (def num-divisions 48) (def ^:private tracks (atom {})) (def ^:private cur-id (atom -1)) (defn ^:private gen-id [] (swap! cur-id inc)) ;; increments cur-id and returns new value (defn remove-track [track-id] (swap! tracks dissoc track-id)) (defn replace-track [track-id new-track] (swap! tracks assoc track-id new-track)) (defn add-track "Add track and return id for it" [track] (let [track-id (gen-id)] (replace-track track-id track) track-id)) (defn ^:private to-freq "Abstraction to convert note name to frequency. This black box hides whether the note name is a midi, or goes through midi, or something." [note-name] (synth/note-names note-name)) (defn ^:private checkbox-to-note [checkbox-name] (let [[note division] (clojure.string/split checkbox-name #",")] {:length 1 :freq (to-freq note) :division (Integer/parseInt division)})) (defn instruments-and-notes [] (synth/instruments-and-notes)) (defn params-to-track "Convert POST request data into a track. Each track is a list of maps, each map contains the key :note and :division." [instrument params] {:instrument instrument :notes (->> params (map (fn [[key _]] key)) (map checkbox-to-note) (sort-by (fn [{div :division}] div)))}) (defn play-all [] (synth/init) (synth/loop-tracks tracks num-divisions))
Contact
Peter Friend (pcf38@cornell.edu), Ian Vermeulen (iyv2@cornell.edu), Haoyuan Chen (hc576@cornell.edu)