Elements of the Connect system
There are lots of moving parts to Connect, both literally and figuratively. This page tries to speed through some of the key elements. We’ll think about:
- Participants
- Mechanisms
- Microcontroller & other hardware
- Coding environment
- Approach to animation programming
- Workshop design and approach
- Limitations, challenges, and ‘what we’d do differently.’
This page isn’t a full write-up of the project, but does aim to capture some of the key thinking that developed over the course of the project. As of this initial writing (June 2024) it’s somewhat stream-of-consciousness. That’s been a useful exercise for us, but likely isn’t as clear as it might be for others.
If you’re interested but overwhelmed or confused, please get in touch.
Participants
Connect workshops were designed for and delivered with children of late primary school age – roughly 9-11 years old. Children worked alongside parents, carers, or other family members (aunts, uncles, grandparents).
The intergenerational aspect was important for several reasons. Partly because we (the project partners) wanted to offer families a challenge which, as they overcome it together, could help provide a context for future family discussions about STEM and STEM study. It’s important for us to build opportunities for those stories with children before secondary school… but younger children often lack confidence or experience with craft and physical tinkering.
Simultaneously, we noted that older family members often lacked confidence with modern approaches to programming, whereas children would typically recognise and have positive associations with ‘Scratch-like’ (block) coding environments.
Where possible, we focused our attention on areas of deprivation, and/or where communities are underrepresented in the STEM workforce, including rural areas. In practice, these criteria cover much of our catchment area in the North-East of England. Where we worked with schools, they were encouraged to offer the workshop to children and families they thought would most benefit from the experience.
A success story
The video clip below shows a delightful cat puppet made by a family in a Connect workshop. The animated element is a coil of emoji-like poop which emergest from the… er… ‘appropriate’ place. The conversation overheard between the family went as follows:
“It just sort of plops out.”
“It does.”
“I want it to, you know, turtle a bit. Come in and out and then plop.”
“How are we going to code that?”
We (obviously) didn’t build Connect around the idea of a family constructing a pooping cat puppet. However, the mechanism, craft, controller and programming aspects all supported – even invited – such an interpretation. The family did, in the end, succeed in animating their cat’s poop. And that’s not a sentence I thought I’d ever write.
Microcontrollers
This is where one would typically write ‘after an extensive search…,’ but in this case the whole project design was enabled by a specific controller.
We’ve long been fans of Micro:Bits – they’re tremendously flexible and well designed. However, since they’re most commonly used in a classroom environment, their radio capabilities extend to local-area communication only. Few UK schools have reliable public WiFi, so Micro:Bits use their own radio system.
For this project, we knew we’d be working with families across a wider geographical region. We wanted internet-based communications – we were explicitly seeking an ‘internet of things’ approach. What we wanted was a board which combined the Micro:Bit’s integrated buttons and display, ease of programming, and the WiFi capability of boards like the ESP8266.
This post on Hackaday alerted us to the Kniwwelino, a Luxembourgish board designed for… well, pretty much our use case.
The lovely folks at LIST have been tremendously helpful, providing the full source of their programming system for us to customise and working with us to secure supplies of the board despite the double whammy of a global pandemic and a specifically UK trade challenge.
Hardware: Mechanisms
Early workshops assumed that devices would be constructed almost wholly from cardboard. However, there were several problems with this thinking:
- Cardboard is largely unfamiliar as a mechanical material. Even the structural use of cardboard is not well understood by participants.
- Purely cardboard mechanisms are at best flimsy and unreliable, and at worst impossible.
- While ‘gears, levers and pulleys‘ are on the Key Stage 2 curriculum, they’re not well understood and few participants would voluntarily construct a mechanism of any complexity.
Accordingly, we went through several design iterations of a ‘mechanism construction kit.’
A ‘sticks, pivots and linkages’ kit was made. Very early prototypes were made from cardboard, quickly followed by lasercut ply, as above. Even this proved too flimsy, and we eventually had to move to plastic.
One particular stumbling block was how to construct pivots and linkages. Very early ideas involved small nuts and bolts, then we explored straightened-out paperclips. A key breakthrough was the observation that floristry wire possesses exactly the required properties: it’s conveniently available in straight pieces (rather than on reels), easy to cut to length by hand, readily malleable, and yet stiff enough to hold a shape.
On one or two occasions, the connection to flower arranging helped workshop participants recognise the activity as being within their range of experience, rather than thoroughly alien.
We still have work to do refining our CAD models, and plan to add in some more esoteric pieces which experience suggests would be useful in practice.
Coding environment
Kniwwelino builds on Ardublockly to provide a customised, web-accessed, block-based coding environment. Source compilation is handled centrally by the web server, with firmware delivered to individual devices over-the-air.
Consequently, no software installation is required for participants; the entire environment can be accessed via a public website. Better still, participants can access the system after the workshop and ‘pick up where they left off’ – their individual device’s code is stored on the server.
Coding environment, in-progress angler fish puppet, and coloured paper: typical ingredients of a Connect workshop.
LIST generously made their entire toolchain available to us, which allowed us to build a very heavily customised experience. Code constructs which were less relevant to our specific workshops could be hidden or removed, and our own blocks and constructs added.
There’s lots to say about this aspect of the system. Some headlines:
- Children recognise block programming (from Scratch work, in school), and have positive associations with it (they’ve enjoyed Scratch previously).
- That this isn’t Scratch doesn’t phase them at all, they just get on with it. Some of the constructs and approaches we’re taking are rather different to Scratch, but those differences seem discoverable and comfortable.
- Compiling Arduino code remotely is quick, but flashing the device takes a minute or two. That delay in the code→run→observe→edit cycle seriously affects participants’ enthusiasm to tinker with the coding aspect. It’s not crippling, but it’s clearly less than optimal compared to the wholly-local approach of, say, Micro:Bit.
Programming Animations
Typically, servo motors are commanded to move to an angle. They’ll then slew to that position at their maximum rate, typically in a couple of hundred milliseconds. That clearly won’t do for expressive puppetry, where you more commonly want sequences of smooth motion, often simultaneously across multiple servos.
The Arduino ServoEasing library offers a partial solution: it gracefully handles smoothed movement, including with easing transforms. Servo operation can be much more organic.
However, building movement sequences with ServoEasing is not trivial. Your main code thread becomes an animation timeline, and you have to manually calculate delay() calls (in milliseconds) to get movements to operate sequentially, and to synchronise the movement of multiple devices.
Most animation systems simplify this process by adopting a project timeline, and applying a keyframe model. Unfortunately, this approach doesn’t map well to a block-based coding environment.
For Connect, we instead took inspiration from behaviour-based animation systems (example: Apple Motion), and implemented a command queue system on top of ServoEasing. Each servo executes its individual queue in sequence. Programmatically, the user can issue multiple movement instructions without having to calculate when each will finish.
Additional queue commands can prompt a servo to pause its queue execution for a time, or until messaged from a (queued) instruction on another servo. Accordingly, it’s possible to synchronise complex movement sequences across multiple servos.
In use, this approach appears to map well to workshop participants’ expectations. Partly, we suspect, because it matches their understanding of how graphical animations work in Scratch. Most (all?) children were comfortable adopting the provided block commands and could manipulate them to work towards their objectives. We’ve more work to do here to test our casual observations, but it seems the programming approach maps relatively well to the natural language participants use to discuss the puppet movement sequences they desire.
This style of ‘implicitly threaded’ implementation is not, we think, common in microcontroller/physical computing systems. Our Arduino library is somewhat scrappy and needs some refinement. As a separate effort, we’re also reimplementing the approach for MicroPython: that work is at least partly functional and we hope to be able to share it relatively soon.
Workshop design & approach
There are a lot of elements to Connect, and the workshop sequence went through multiple iterations in the early delivery phase of the project. There are several insights, some of which are possibly controversial even within the project team. In no particular order:
Mechanisms are hard. They’re unfamiliar, and few participants have much mechanical intuition on which to draw. Prior experience with other construction system (Lego Technic, for example) doesn’t appear to offer much help: with rigid systems you don’t get to explore the advantages of straight linkages, they’re simply imposed on you. Add in the unfamiliarity of servo motors, and already a lot is being asked of participants.
‘Puppets’ are a familiar concept, but unfamiliar artefacts. Everyone’s seen puppets, but few have thought about how they’re made or operated.
Progressive reveal. In some early workshops we tried to describe the destination right at the start. That never really worked, because there were too many unfamiliar concepts in rapid succession. We ended up starting from mechanisms, then extending those to mechatronic puppets. Once participants had something moving we’d introduce the coding environment, and the networked aspect was something we mostly let participants work out in their own time. “Build an internet-connected puppet” was a useful phrase in marketing the project, but it appeared in the workshops in the past tense.
Constraints are good. This will be familiar to any facilitator of making: start from an open brief, and participants’ aspirations are often driven by their imaginations rather than practical constraints or available skills. That is: they’ll dream something they can’t build.
For the workshop designer and facilitator there’s thus a tension between wanting to build participants’ skills, and supporting them to adopt more achievable objectives. For Connect, the project team initially worked to scaffold understanding: we piloted activities to support learning about different mechanism geometries.
Eventually, we ditched most of that as being too abstract, and let participants explore and fail. The key tool for facilitators was a pencil, to quickly sketch possible solutions to the problem the participants had now identified. As well as providing just-in-time, on-demand support, this also modelled an appropriate communication approach for participating families; through the workshop period, many became gradually more free to sketch their point, or quickly cut some paper to illustrate it.
Parts, Purposes, (Complexities). One part of the scaffolded learning approach which we retained was borrowed from Harvard University’s Agency by Design project. When participants enter the workshop, their workspace includes a very basic mechanism, which is already moving. Their initial task is to draw that artefact, and identify the purposes of its components.
We find this useful for gaining familiarity with new materials; setting a context for how those materials will be used and establishing a shared language for the components (‘servo motor’, ‘linkage’, ‘pivot’, ‘support’, ‘controller’, etc.). In our application, we found the ‘Complexities’ terminology less helpful: our participants haven’t yet established the knowledge or experience base to form critical judgements in this space.
Thinking in three dimensions is tough. We sketch in two dimensions, and many participants’ experience of crafting is similarly constrained. Add in unfamiliar materials, and many participants find themselves understandably ‘blocked’ when trying to, for example, orient a servo motor: they’ll rotate it around one axis, but not the other two.
Accordingly, most Connect puppets – and many of our examples – are inherently planar: they’re a fundamentally 2D design that’s been cut out. Even then, a degree of depth is required to overlap components as they move. Pragmatically and by inspection, this is about the right level of complexity for most participants, and for the available workshop time. That said, some participants built much more complex dimensional artefacts.
Children can concentrate. Our original plan called for multiple hour-long workshops delivered over consecutive weeks, but project constraints (coughpandemiccough) caused most deliveries to be single workshops of around 4 hours duration. This was undoubtedly challenging and tiring for participants, but (once refined) the format was more successful than we’d expected. Anecdotally:
- The rate of progression was relatively rapid.
- New ideas were (in these later workshops) introduced gradually and sequentially.
- Hour 1: new ideas, materials, thinking. Introduced while participants were fresh and keen. Peak ‘unfamiliar’?
- Hour 2: familiar craft work, realising the participants’ own design: peak ‘agency’?
- Hour 3 (after lunch): context shift to coding, which for most children felt refreshingly familiar. Peak ‘comfort’?
- Hour 4: Integration, networking, race to finish. The new ideas are clarified as they come together. Peak ‘delight’.
Oh. I seem to have accidentally written a framework for computational tinkering activity design. Look at that. Thanks, brain. Next challenge: see how well that holds up to inspection. My immediate guess would be ‘not very well,’ but it’s going to be interesting exploring that.
There’s a heap more to say about this area. Not least, trying to unpick which aspects were Connect-specific, and which are of more general utility.
Smaller components
Anyone who’s read this far will have gathered that there are many moving parts to the Connect project. Here are a few more:
Mobile networking
For Connect to work, the microcontroller for each puppet needed a WiFi connection to the world. We also needed laptops or similar with wifi access, and that part is easy: every school we visited had a class set of iPads or laptops we could use, and they knew which rooms had network coverage. The Kniwwelino microcontrollers, however, need a straightforward ssid/password network: there’s no system for clicking ‘accept terms and conditions’ or anything like that.
Accordingly, we chose never to rely on school wifi for the microcontrollers, and instead carried around a 4G mobile wifi gateway. It’s surprisingly hard to work out many wifi devices these gateways can support. In some cases it’s limited to only half a dozen or so. Out Netgear Nighthawk MR2100 could handle 20, we think?
In practice, this worked fairly well. We often needed external antennae for the Nighthawk, and more than half a dozen Kniwwelinos all trying to do OTA firmware updates over a slightly edgy 4G signal was not ideal. It mostly worked, however.
On one occasion we struggled with both venue WiFI and 4G signal, and ended up using a miniature WiFi router hanging off a stray ethernet cable. This was probably rather cheeky, but it worked.
Emergency local system
One significant concern was ‘What happens if there’s no wifi or mobile signal on the day?’. Our answer to that was to run the entire server stack on a Raspberry Pi. I’ll spare you most of the details, but we ran a WiFi Access Point, a web server, the coding environment, compiler toolchain (which was… interesting, as cross-compiling ESP8266 from ARM was… well, it eventually worked), and MQTT broker, all on a single Pi. It wasn’t even slow, as such: the first compile for any individual Kniwwelino took a little longer than I’d like, but subsequent compile times were more than adequate.
Thankfully, the Pi server stayed in the kit box throughout the project – it was never really tested with a full workshop load. I’m fairly confident it would have worked, however, and it offered a degree of comfort simply for existing.
Telemetry
At startup, individual Connect puppets announce themselves to the central MQTT broker. We built a small logging/dashboard application which monitors individual devices and notes when they were last seen. We (deliberately) don’t know which device was with which family, but we can see in which cohort/workshop group the device was distributed.
Data analysis is very ad-hoc still, but it’s clear that:
- A significant proportion of puppets were brought back online after the workshop, therefore presumably at home.
- Many, however, weren’t. We think this was partly the result of an update to Samsung mobile phones which broke a critical part of the workflow: the fix suggested here would likely solve the issue at our end, but we weren’t able to incorporate it during the initial project. It’s a reminder that systems like this can be fragile.
- Some puppets remained online for a considerably period post-workshop. In some cases, continuously for weeks or months. In others, sporadically, for a year.
- In theory, we’re also logging when the devices have been updated. So we could probably count how many were re-programmed after the end of the workshop.
There are some exciting possibilities raised by the networked nature of the project, most of which remain unexplored for now.
Numbers, gender, venues
This isn’t the place for formal reporting, but as an outline: Connect worked with 160 families. 49% of children and 59% of participants overall identified as female.
We delivered 31 workshop in total. In the aftermath of the pandemic, some sessions were sparsely attended and most were run with reduced room loadings. Later sessions were typically well attended, but with the usual variability that comes with free public events.
Venues included the Life Science Centre in central Newcastle-upon-Tyne; primary and middle schools, across the north-east of England; libraries, including several in relatively rural settings; a community museum; and a community centre working with vulnerable children and their families.
Most deliveries were single 4-hour workshop. Other formats included a weekly course of 4x1hr after school sessions, and a 2×2 hour ‘afternoon’ approach.
The vast majority of participants (and all those counted, above) completed the course or workshop. Exceptions were related to illness, or where participants were challenged by the workshop format or context – typically around group-related anxiety or noise levels.
Moods and emotions
Connected devices – an Internet of Things aspect – was part of the project plan from the very start. We knew we wanted to work with families, mostly in informal learning environments, and that extending the facilitated workshop experience into a self-supported home environment was a key objective. Having the devices communicate between families seemed like an obvious step: these days everything is connected, why not the things one makes?
What we didn’t want to do, in this circumstance, was accidentally run a social network. Devices would conceivably end up in children’s bedrooms, and building a system which could send and receive arbitrary messages was clearly unwise.
As is so often the case at such times, the solution came from accepting limitations. The Kniwwelino microcontroller has a 5×5 array of single-colour LEDS. While you can scroll text across that display, the result isn’t very readable; it’s much better suited to displaying small, low-fidelity icons.
The most recognisable of these we termed ‘happy’, ‘sad’, ‘heart’, ‘skull’ and – at the request of participants in a pilot workshop – ‘silly.’ And that was it. That’s the full range of messages Connect devices can process. We typically referred to these as ‘moods’ – a less loaded term, we thought, than ’emotions.’
For participants, that meant thinking about their puppet, and how it would move to convey some or all of those moods.
This ‘felt right’ for technical and pragmatic reasons, but also because we were trying to offer a workshop which appealed to creativity rather than competition.
Perhaps unexpectedly, some participants – and more particularly schools and host venues – reacted very positively to this emotional context. There are, we think, several reasons for this. We hope to explore them in due course.
One aspect is clear, however: there’s demonstrable appetite for digitally-mediated activities which aren’t competitive. In the UK (and internationally) we have several schools’ robotics competitions. Some of them are exceptionally well run, and both schools and participants love them. Not all children, however, are motivated or enthused by competition.
Limitations, missed objectives, regrets, unexpected wins
Oh, where to start? In note form (and from largely anecdotal evidence)
- Powering servos is a nightmare. We ended up hand-building something akin to a jump lead, and hard soldering it to each Kniwwelino. The resulting preparation time was unpleasant. We wish we’d had more ability to customise hardware, even at the trivial level.
- Workshops of this sort are facilitator-intensive. As a result, the project overall was relatively costly. Is that likely always to be the case for digital tinkering?
- This level of integration of mechanism, physical build, computer control and networking was delightful, but also… a lot for people to get their heads around. Participants, but also for the project team and our colleagues. We’ve several thoughts about other activities and workshops we could run which might better suit different delivery environments.
- Connect grew out of previous workshop activities and installation pieces we built for Maker Faire UK. We’d like to revisit some of those in the light of what we know now.
- One (unexpected?) finding: some participants found the buzzing noise of servo motors unbearable, and had to leave very early in the activity. We failed to find mitigations for this, other than offering alternative activities for the families impacted. In some respects, having devices moving (and hence making noise) as the participants arrived at least clarified the circumstances of the workshop.
- We never really worked out what an ‘example puppet’ should look like, and we didn’t manage to carve out as much time as we’d hoped to just play, ourselves, as the project team. We had examples, but many were only partially finished. Others looked good but didn’t work terribly well. A few worked well but looked rather scrappy – we mostly ran with those in workshops, because ‘I could do better than that’ is a good place for participants to be. But it does feel like we haven’t explored the limits of what the system can handle. We’d love to have somebody like, say, Rob Ives join us for a week!
- The original project plan was to regard the workshops as a research context alongside delivering the public project. For practical, pandemic-related reasons that didn’t happen; we’re currently planning to host more workshops which we can properly instrument for research purposes.
