Another Project?
Yes. Another. Project. And no, it’s not yet another passenger car lighting project! I can hear you rolling your eyes from here…
This time it’s about adding sound to anything that you please using an Arduino ESP32-S3 Super mini and an I2S amplifier. But I’m jumping the gun here going on about the specifics already. Let me start at the beginning, with a little history on our “sound projects”…
A Little History
We started looking at “sound enabling” our equipment years ago. January of 2020 to be exact. The idea was to couple an Arduino and one of those “cheap” MP3 player modules as a low cost sound option. When compared to the cost of retail sound offerings available, often for hundreds of dollars or more, the motivation is obvious.
Long story short, those “cheapo depot” MP3 DFPlayers turned out to be absolutely unreliable. Nothing but junk. No amount of tinkering or tweaking could guarantee operation without having to “ruthless reset” everything. We ended up mothballing the project indefinitely. Well, at least until I could find a more reliable solution.
It was quite an effort to get as far as we did, but it left us with a reusable software implementation based on the concept of a “playlist”. The backend (Arduino) serves up the playlist in JSON format to the front end, a browser based user interface (UI), served up via WiFi from files stored on the backend.
New Found Motivation
Enter the idea of the I2S interface. Not the I2C interface. I2S, short for Inter Integrated circuit Sound, developed by Philips back in the ’90s for stereo equipment interconnections to share audio streams between components, both internal and external. The more I read about it, the more I was convinced this was the solution I was looking for.
Being the electronics and audio buff I was growing up, I’m surprised I hadn’t heard more about it at the time. Late to the party, I found some folks online using various I2S audio decoders in their projects and settled on the MAX98357 based decoder. It has a built in 3W amplifier, with a somewhat odd 0-21 volume range, but whatever. At least it works!
But I’m skipping ahead again. While the Maxim hardware does everything for us, it requires a LOT more horsepower from the Arduino to handle converting and the streaming the audio data to the decoder hardware. When I say a LOT more horsepower, I mean at least an ESP32-S3 (dual core) to handle the load.
Time To Order Parts
I’ve never used an ESP32-S3 so I don’t have any to experiment with. Don’t have any of those Maxim decoders either, seeing as how I’m just now reading about them. While the Arduinos I’ve been using aren’t a dime a dozen, they aren’t ten dollars a piece either. They’re roughly twenty dollars for three of these “high horsepower” ones.
The Maxim decoders are about five for fifteen dollars. The only thing missing is an SD card reader. They may make an Arduino package that has one already included, but I haven’t looked for one (yet). I found five for $7.50. So far grand total of $7 + $3 + $2.50 = $12.50! That’s not including the micro SD card to store the sound files on.
That’s an order of magnitude (10x) less than the retail price of a low end sound system! Last time I bought a 32GB micro SD card was years ago, but it was ~$7.50. So let’s say $20 vs. $200. Sold! Thinking about it, maybe there’s a version with enough flash storage onboard to eliminate the need for a card reader altogether. Depends on the number and size of the files I guess.
The next step is to bring that old MP3Player project out of mothballs and see if we can breathe new life into our aspirations for sound.
What’s Next?
Time to start working the old software into a new project to meet our needs. The old project is one of the older ones and doesn’t have any of our latest improvements, like themes and OTA progress indication. But the first thing to do is get rid of the DFPlayer stuff, then update to using LittleFS in lieu of the old SPIFFS library.
I’ll try not to get into too much detail about the software, but that’s what we’re talking about, so… If you’re not familiar with Arduino speak, the Serial Peripheral Interface Flash File System (SPIFFS) is deprecated. The replacement recommendation is the Little File System (LittleFS). This is used to read files like the HTML web files (internal flash) and sound files (SD card).
Those libraries are “built in” to the specific Arduino hardware platform, unlike some of the others we use, like ArduinoJson and I2S_Audio that perform specific tasks regardless of platform. Fortunately, this sketch doesn’t need much else beyond those and the WiFi libraries.
Eventually the code we use to encapsulate the audio functionality will have its own library, but for now it’s part of the sketch to ease debug. Thankfully most of the playlist handling code doesn’t need much modification. Really the only “major” change is implementing looping in software where it used to be hardware driven with those junk MP3 modules.
Sounds Easy Enough…
The software is already pretty close, at least the UI code is, sans the equalizer settings. The looping is “commented out” until the backend Arduino code gets updated to implement it in software. The messaging piece is already there, it’s just adding the logic necessary to emulate what the old hardware used to do for us automatically.
The neat thing about the existing backend code is it already responds to event messages from the old hardware when the previously played track finishes playing. It should be easy enough to substitute the event processing for the new hardware, tracking the current looping mode to determine which track is next, if any.
Most of the effort on the UI is getting everything updated to use the latest improvements, which involves comparing the older existing code with current versions from other projects, like OiffceLighting, LithoPhaneLEDs, and PassengerCarLighting2812. We added two new themes for PassengerCarLighting2812 recently, highlighted in the Latest Evolution Lighting post.
Development Concerns
While we’re talking about adding themes, that brings up another “issue” with using Arduinos, at least when it comes to uploading files to them. The Arduino IDE, with the proper uploader installed, will upload all files in the “data” folder to the flash file system on the Arduino. No problem there as long as you can find and install the proper uploader package.
Right now each and every project has to have its own “data” folder with a “themes” subfolder in it. Any change, or in this case addition, in any one project must now be manually propagated to every other project to keep everything in sync. Similarly, all the CSS and javascript files suffer the same fate. Basically any shared common files.
There may be a way to use symbolic file links in place of the actual files, but I haven’t tried it yet. Hopefully the IDE won’t have problems with the file system interface using symbolic links. If that works, it will allow one set of files to be maintained and all projects can link to that “master” set. But let’s get this deployed before we go mucking around with something else…
Some More History
Just a little more. I promise. If you want A LOT More History, I’ve tacked it on the end of this post.
We started on this latest adventure going on six months ago as of this writing. But I let myself get sidetracked with other projects, Hurricanes, etc. You name name it, I got distracted by it. Some of those distractions were acquisitions and the ensuing investigations of their installed electronics, like battery power and sound systems.
The I2SAudio project was put on the back burner for at least a few months while all that was going on. But all the basics were there and working. We’re able to play sound files stored on the micro SD card from the UI. Only dynamically building out the looping buttons based on the playlist was left to do.
Past tense. That’s done now too. That leaves expand/collapse of the playlist in the UI. I swear I have code somewhere from past experiences with a javascript/CSS based “twisty” implementation. I’m hoping that I can leverage it, but like most things in my favorite place, somewhere, it will never be found quickly nor easily.
Making More Work For Myself
Defeated by wasting time looking for it, I tried yet another internet search. Unlike the first few searches that turned up nothing useful, encouraging me to look for that old code, this one came back with a new HTML5 construct I didn’t even know existed. The <details> and <summary> tags provide a built in expand/collapse function without the need for code behind it.
At first I thought this was the quick and dirty way to get there from here. WRONG! The entire playlist tree in the UI is built dynamically from the playlist data received from the backend. Using nested HTML <ul> and <li> elements, we build the hierarchy one entry at a time. Each <li> element has a <span> that contains the actual “name” of the entry in the tree.
All these elements are given a specific CSS class to correctly render in the browser. Without going into too much more detail, it’s important to note that any selection clicks made in the UI on any of those elements in the playlist tree get “bubbled up” to the top level element, in this case the top level containing <ul> element.
Why does this matter? Because now in the javascript event handler we have to figure out which one of potentially hundreds of elements was clicked on, then backtrack and modify the CSS class of the affected element(s) to indicate selection visually. Before adding the new elements, this boiled down to one of the three element types already mentioned. And it worked. Past tense.
Why Did You Break It?
After modifying the javascript dynamic generation function to add the new <details> and <summary> elements, all of a sudden NOTHING WORKS! So here we are weeks, if not months later, still fixing the “quick and dirty” way. But at least it’s working now. Even the CSS needed modifications to work with both versions.
I commented out the old, working version of code as I progressed with the new version. Boy am I glad I did! I went back and made it selectable using a single flag to indicate whether to use the old or the new version. My thought is maybe I’ll come back to this some day and figure out how to make it all work without the newer HTML5 constructs.
So why in the world would I want to come back and redo this, potentially breaking everything yet again? Looking back at how each level of the hierarchy needs more and more of these <details> <summary> blocks, it’s apparent these were meant to hide large blocks of HMTL, most likely upper level <div> element containers, not each and every nested element of a tree.
I’ll spare you and save those particulars and details for later. At this point, the only thing left to do for the existing UI is figure out why it stops selecting down to the lowest level for all child items except the first child of the parent selection. What did you just say? A picture is worth a thousand words…
There, I Fixed It
Well, not really. That can wait until we know this is really the UI paradigm that best fits what it is we’re trying to accomplish here. What you see is essentially an MP3 player interface, with play, pause, stop, next, previous, etc. There’s also a set of looping selections, based on the top level selection, in this case Genre, Artist, Album, or Track.
But those “classifications” don’t really fit either. Function type seems more suited for Steam Whistles and Diesel Horns than Genre. Not sure how to classify Single Chime vs. Three Chime, but Artist isn’t it. Road Name seems a better fit than Album for C&O vs. CN. Track may fit, kind of, but I think you get the idea.
The good thing is these classifications are all easily configurable as part of the playlist. But keep in mind for every new classification, there must be a corresponding addition to the CSS to accommodate it. The other thing to keep in mind is they can also be directly tied to the currently selected theme settings, like the “Teal Mobile” theme used in the above screenshot.
A New Look?
Regardless, the UI is going to need a facelift moving forward. Now what that should like like, I have no idea. Guess I need to look at some of those sound systems on the market to get an idea what theirs looks like. But even then that may not be the best fit for our application.
When it’s all said and done, we have everything we need in place whenever we finally figure out what it’s supposed to look like. Most of the commercial products I’ve seen tend to break up sounds into an intro, sustain, and ending portion, stringing them together, starting with the intro, then looping the sustain part, and finally play the ending to finish up.
Not sure how “quill” will work, but that’s a problem to solve another day. I’m sure there are other “features” we’ll need to add as we go, but for now we have a reliable, extensible solution. With that in mind, I’m wondering if we have the capability to play two tracks at once.
Stuck thinking in that MP3 player mindset, I just issue a stop, then play a new track. Wonder what happens if I don’t stop the currently playing track? I did a deep dive into the library code already to see for sure what was going on for other issues I ran into, but it would be easier to just modify my code and see what happens. Probably a kernel panic!
A Look At The Hardware
Originally just an ESP8226 was used for the MP3Player project, more than adequate to communicate with those POS DFPlayer modules over a 9600 baud serial connection. But even that simplicity was unreliable, or rather, the modules themselves are unreliable.
Fast forward to now, using I2S audio technology, proven over decades. But there’s a price to pay when it comes to the control side of things. Beyond the extra horsepower, the full sized ESP32-S3 is a sizeable piece of hardware, consuming nearly an entire half sized breadboard. In fact, to have space for anything else, we used two of those half size breadboards side by side!
The ESP32-S3 module itself is stout enough to hold the two breadboards together once plugged in. There are only two other components required, but to be sure there was space for them AND the connections without having to run half the connections under the Arduino, two side by side are better than one.
The full sized S3 module is labelled with an abbreviation of the project and its last IP “triplet”. This is absolutely necessary for me to keep my sanity. When “rooting” through a box of old projects with probably a dozen Arduinos inside, it’s maddening having to plug in each and every one just to see what it’s running and what IP to use in the browser.
The two purple modules are the actual I2S decoders, one for the left channel, one for the right. Notice only one is connected. That’s because the decoder can also combine both channels into a single mono output as well. Good enough for what we’re doing. The actual formula is (L + R)/2. The mode is selectable with a single input, left, right, or both.
The blue module standing up next to the speaker is the micro SD card reader. All the audio files are stored on a 32GB micro SD card. The S3 reads the files and converts the audio to an I2S stream that the decoders turn into audio output to the speaker with over 3 watts of audio power! More than enough for our needs.
The speaker is a leftover from one of our Bachmann Ten Wheeler projects. We replaced the old tender bottom half with a new one, complete with “updated” electronics for chuff sound. Still the same tired old 9V hiss generator connected to a chuff switch… That only activates twice per revolution. Not very realistic, but at least the new red painted wheels look nice.
In any case, the speaker in the old bottom half became a “spare”. IIRC we ended up with at least three if not four “spares” this way. I designed and 3D printed the speaker enclosure, hoping to give the tired old 1W 8Ω speaker a fighting chance to sound good. It helps, but with its limited response, there’s not much else that can be done.
Something’s Different
You may have noticed a difference between the full sized S3 and other Arduinos, the number of USB connections. One is the standard Program Upload and Serial Monitor port. The other is for JTAG debug. JTAG is short for Joint Test Action Group, which defines a means of getting “inside the hardware”, essentially allowing the equivalent of attaching test probes inside the chip.
Apologies, that’s a very dumbed down explanation of a fantastic innovation in hardware development, also proven over decades. We won’t need that low level debug ability. At least, not yet. More pertinent to our discussion is which is which? The one that works is the one we’re looking for. Just that simple. The other is enabled and configured using the board settings.
The obvious difference with these “Super Minis” is their size. They’re so small it presents a problem when labelling. There’s no big metal can to stick the sticker on. Haven’t tried it yet, but probably have enough room on the back for a sticker. They also run hot to the touch. Hotter than their full size counterparts for sure. Beware.
That’s Not The Only Difference
Another difference with these S3s, and even the ESP32-C3 Super Minis, is the actual board selection. They only have one USB port, but good luck finding the correct board for your particular flavor of Chineseum. Nick had ordered a set of the C3 and S3 Super Minis and gave me a sample of each.
Even though the silkscreen on the C3 says it’s a specific type, selecting that type in the Arduino IDE board manager does not allow me to communicate with it. It also causes build errors that go away when selecting the generic ESP32-C3 Dev Module. What’s interesting is there are contact pads on the PC meant for “pogo pin” connections.
The C3s I bought have more contacts than the one Nick gave me. It’s a crap shoot when dealing with any of these “dime a dozen” Chineseum parts regardless of where they’re sourced. It would be helpful if the competition to mass produce knockoffs of knockoffs to make of profit didn’t make it impossible to get anything more than basic information, let alone good information.
That’s the nature of the beast. It would be different if paying more for quality actually worked in this case. Been suckered by that before with these things. About the only way for me to know is if they can be trusted is to order from an Amazon seller I’ve dealt with in the past that I feel can be trusted. Sometimes even that isn’t enough. Caveat Emptor.
What’s A “Pogo Pin”?
These were designed in the ’70s for testing integrated circuits and other micro circuits. Basically they’re a spring loaded pin the provides positive pressure on a mating contact pad. I hadn’t seen them is common use until using a Raspberry PI Zero. The I/O is limited on the PI Zero, but an expansion board can provide more USB connections.
The expansion board “piggybacks” on the original PI Zero, making connection via pogo pins on the expansion board that mate to a set of built in pads on the PI. These Super Minis are the only other commercially available products that I’ve seen them on.
That’s not to say you have to use pogo pins to make the connection. It’s just another solder pad, it just doesn’t have the plate thru hole. You can just solder wires directly to them, but be sure to strain relieve the wires. Don’t depend on the pad to hold the wires in place or you’ll find yourself in the situation where the pad got lifted off the board and went with the wire.
The Particulars
If you’re already familiar with HTML, feel free to skip over this. For those unfamiliar with HTML, it’s basically a page description “language”, a means of describing a page of text, images, graphics to a browser and how to present the various elements of the page, even dynamic elements like Data Driven Documents (D3). We use that to display current sensor data for our block controller.
With Cascading Style Sheets (CSS), we can tell the browser what colors to use, what font/size/color, whether to left, center, or right justify, etc. for every element! Javascript adds another dimension of dynamic interaction between the page and browser without the need to reload everything from the web server for every action, even override the default event handling.
We’ve mentioned a number of HTML elements used. I think the <details> and <summary> tags are self explanatory, but one item of note is the <details> tag has an “open” attribute that, when set, expands to display everything between the <details> and closing </details> tag. Otherwise, just the items between the <summary> and closing </summary> tag are displayed.
More Details
A division tag, or <div>, is a sort of container, meaning everything between the <div> and closing </div> it to be treated in a similar fashion. Generally it will have a CSS class or classes associated with it, sometimes even an id. The corresponding CSS describes how to present these items to the browser, even whether to display or hide the entire division.
The unordered list <ul> and list item <li> tags are a means of grouping similar items together, like a list of bullet points. CSS can describe exactly whether you want bullets, squares, triangles, or nothing at all in front of each list item. Unordered simply means it’s not a ordered (numbered) list <ol> type, which has it’s own styling settings.
Each list item can have its own child element(s), like the <span>, or even another unordered list, with its own list items, in nested fashion. Each level in the hierarchy can have its own presentation style described with CSS. For example, present the highest level items with a light red background color, the next level down as light yellow, the next light green, the last light blue.
Even better, CSS can tell the browser to invert the item when “moused over”, swapping the color of the background and the font, without having to write javascript event handler code to monitor for when the mouse is actually over the item. Same with currently “focused” items. More complex CSS “selectors” can provide finer control over browser actions.
Developer Tools
If you’d like to learn more about HTML, CSS, and javascript, there are plenty of resources available online for free. For example, W3 Schools, has tutorials for just about anything you’d like to learn about coding. I find it to be an invaluable tool for HTML, CSS, Javascript and jQuery, Python, SQL, etc.
If you’d like to see what’s happening “under the hood” of your browser, try opening the “developer tools”. Each different browser has its own way to enable this mode, but in Chrome, my browser of choice, simply press the F12 key. I use it to monitor and debug my code ALL THE TIME!
Here’s where I spend most of my time when adding and troubleshooting new features. The screen shot shows the developer tools inspecting the javascript “Console” log. Starting at the top, it captures the tail end of the Connect message response from the backend, including the playlist and themes data used to build the UI presentation.
Below that is the beginning of parsing the JSON message into actions and dispatching to those specific action handlers. The repeated “WebSocket.js:254” message demonstrates the looping behavior as each new action is dispatched, in this case power, then version, then volume which is dispatched to its corresponding handler.
It lives at “WebGenAudio.js:108”, or line 108 in the WebGenAudio.js file. This handler was “registered” with the main WebSocket.js handler for all web socket communication between the front end (UI) and the backend (Arduino). The power and version messages are common to every project because they all have a Power button and a software version field in the Settings.
Not every project uses audio though. All the audio related processing is contained in the WebGenAudio.js file. Similarly, all themes based processing is contained in the WebThemes.js file and again, action handling for themes is registered with the main WebSocket.js web socket handler.
This amount of information ion the console log is obviously very verbose. Not to worry, there are a number of predefined “log levels” that are turned on or off with the flip of a “switch” at the top of code. Turn them on as needed to troubleshoot then turn them off for “production” code.
Styling Tools
All the audio related styling is contained in audio.css rather than clutter the main.css file with styling only useful to audio projects. The themes module has a livery.css file to describe the various color choices available. It is included by the main.css by default, available to all subsequent styling files.
This is deliberate and part of our modular architecture. If you want it, include it. If not, don’t. Themes is really the only “baked in” choice, but if not wanted or needed, a default livery.css file can be provided or its inclusion in main.css removed altogether.
The developer tools allow you to view all the HTML elements AND their associated CSS styling in the HTML “Elements” tab, just to the right of the javascript “Console” tab. The secret magic trick to using Elements is the “select” tool at the very left of all those tabs. Click it to enter select mode, then go and click on the element of choice and it will take you to that part of the document.
If you don’t make a selection then wonder why clicking isn’t working the way it should, it’s because the browser is still in select element mode. Click the “select” tool again to exit selection mode. Don’t ask me how I know…
CSS Selector Tools
Earlier I mentioned CSS “selectors”. Essentially these are a shorthand for telling the browser, when you see this, do this. For example, we generally assign the “button” class to a button declaration in the HTML source file. The corresponding selector would be “.button” in the CSS file.
If different buttons have a different id, e.g. id=”play” vs. id=”pause”, then “#play” and “#pause” would be the corresponding CSS selectors. These are simple examples, but there are much more complex selectors available, like providing a visual “highlight” when the mouse travels over an element or dimming a button when it’s disabled.
We use these constructs and even more complex ones to transform those awful looking default “radio” buttons like like the classic car, old style radios with mechanical pushbuttons. We even override the awful “File Upload” to match all our other button styling. And it’s all done with a CSS definition. And it’s all handled by the browser! NO OTHER CODE REQUIRED.
The developer tools Elements above displays both the selected element HTML, and its associated style information below in the “Styles” window, including the selectors used to determine how to present it. You can even poke and prod values into the element style to play with various settings without having to edit, save, and reload the page. Very powerful and I use it quite a bit.
A LOT More History
We started looking at “sound enabling” our equipment years ago. January of 2020 to be exact. The idea was to couple and Arduino and one of those “cheap” MP3 player modules as a low cost sound option. When compared to the cost of sound offerings available, often for hundreds of dollars more, the motivation is obvious.
If I can throw together something acceptable feature-wise for a handful of dollars, it’s a no brainer. The emphasis here is a fairly limited feature set. The cheapo depot MP3 player modules are meant for one thing, and one thing only. Controlling an MP3 player with a minimum parts count.
It provides a built in amplifier with 30 level volume control to directly drive a speaker, and it is LOUD! It also has a micro SD card slot for storing the music files, inaccessible from the Arduino using a dreadfully slow and antiquated 9600 baud serial connection, and a minimal interface for standalone operation via a handful of external components. And they’re JUNK!
Originally just an ESP8226 was used for the MP3Player project, more than adequate to communicate with those POS modules over a 9600 baud serial connection. I remember owning a number of 9600 baud modems back in the ancient times, but seriously, that’s how bad those modules were to work with.
If you’ve ever lost a 9600 baud modem connection back in the day while downloading a file, and had to restart all over again when it happened, you know how unreliable they are. Same with these modules, with the same lack of warning.
ABSOLUTE JUNK!!!
That’s right, these DFPlayers are ABSOLUTE JUNK!!! After spending months on end working with these things, it was apparent they would NEVER run reliably. If there is a way, I certainly couldn’t find the magic combination of software and hardware to make them work without constantly getting into a “zombie” state.
Once they’re “zombies”, the only way to bring them back to life is a hardware reset. It appears that the digital noise from the Arduino side of things is just not conducive to reliable operation. I never tried them in stand alone mode, but then again, that’s not the mode of operation we require. I’m not building a “Walkman™”.
It wasn’t all wasted effort though. I did eventually manage to create a workable user interface built around their feature set, and some of those features are rather impressive for the price, like built in equalizer modes, e.g. Classic, Rock, Jazz, etc. Another useful feature is the looping capability. Now if they just worked…
Mothballing The Project
It was quite an effort to get as far as we did, and it left us with a reusable software implementation based on the concept of a “playlist”. The backend (Arduino) serves up the playlist in JSON format to the front end, a browser based user interface (UI), served up via WiFi from files stored on the backend.
The UI relies on HTML, CSS, and javascript to render a modern presentation. The playlist is presented in a hierarchical “tree” fashion, e.g. Genres, Artists, Albums, Tracks. The front end communicates with the backend via a “Web Socket” connection, essentially a bi-directional communication channel that “sits on top of” the HTML connection.
But unlike the HTML connection, immediate updates to the UI can be made without having to “refresh” the page and wait for everything to load, all the files served from the backend again. A “click” to play a track from the playlist in the UI is sent via the web socket to the backend, it starts playing the track, then responds to the UI to keep things in sync.
Want To Know More?
If you’d like to know more about this or other Barkyard RR projects, leave us a comment! You’ll need to create a user first though. In this day and age, it seems like every grifter and con man is constantly spamming sites to the point where we’d spend more time moderating the comments than doing anything useful. We hope you’ll understand.
The only way we have to get around the spam is to ask for a verifiable email address. Until you verify your address, your account remains in limbo, so to speak. We’re not selling your data, that’s not why we’re here. We’re not “monetized” in any way. But if you’re concerned, please refer to our privacy statement.
If you’d like to see more of this type of content, please let us know. If you’d like to get a look at the sketch code or the web page code or need links to what parts we’re using or just have a general observation, please, feel free to drop us a comment. We’ll get back to you as soon as we can.
If that isn’t your cup of tea, then use the Contact Us page. You’ll still need a valid email address though, especially if you want a response! This will take longer since we have to actively moderate submissions vs. the automated user account and email activation handling.
Thanks for following along. Stay tuned. We’ll have more soon!