Braille Lego and Programming for the Visually Impaired

Blog post from Steve Mullarkey Course Coordinator – Level 3 Software Development and Games Programming from Shipley College

The Software Development and Games Programming course at Shipley College is constantly being updated to keep it current. One aspect of this was the increased use of visual programming tools, such as Scratch, as used in Code.org’s Hour of Code activities and Blueprints, as used by Unreal Engine 4, a state of the art game engine.

The students have all reacted very well to these new tools and this year we had eight students leave with D*D*D*, which is the highest grade possible.

However, last year we had a new student who wanted to join the course who is visually impaired – he has no light perception whatsoever. This makes anything which uses drag and drop inaccessible, as his screen reader software, Jaws, can only work with text.

I had to make a choice then;

  • do I say that he can’t join the course?
  • do I change the course to suit his needs?
  • do I somehow differentiate the material so he can access it?

At Shipley College we strongly believe in inclusion and so the choice was a simple one to make, but very hard to execute. We tried different approaches to find out what worked best for Callum.

Fortunately my wife has many years of experience working with assorted special needs students and gave me some great ideas on where to start.

For the scratch projects, I used laminated prints of the scratch nodes, glued to foam board, with Braille added manually using craft supplies and QR codes added to the back of each node – the goal has always been to make the resources so that Callum can work independently.

This worked really well, but it took a over a week to create the resources for an activity that took an hour. Obviously I had to find a method which I could automate.

I knew I could create 3D models of the nodes using code and started to think that 3D printing would be a possible way to automate the whole process. To test this we got the cheapest 3D printer we could find on eBay – this cost around £160 and came with no instructions but it eventually printed a 3D model of a Scratch node.

The challenge then was to add Braille to each node, along with text so that Callum’s LSA could also read what the node was to be used for. The solution for this was to print an inverted V, with text on one side and Braille on the other. To make sure they could be read the same way, the text was also inverted.

Braille-node

Braille-node

This was then tested by Callum and another VI student, who both said they could read the text and the Hour of Code activity worked really well. The only issue now was the reliability of the 3D printer. We needed a professional 3D printer, but we did not have the budget for this.

This is when we saw the Accessibly by Design competition on the JISC Elevator. We quickly made a video and entered the competition, the prize of £5000 would have made this project viable and was something we desperately needed.

We were overjoyed when we found out that we had won funding. Besides the funds for the printer, JISC organised four development sprint days, hosted by each of the winners. These really helped focus the development of our ideas and helped get our students not only involved in the project, but to also have valuable experience, first hand, of real development meetings. The first thing the students did was vote on a name for the project and selected ‘Project ViP’, for project Visually Impaired Programming. We managed to get the projectvip.info domain for future use too!

Working with JISC, students from Shipley College’s year two Software Development and Games Programming course worked on the design and development of the Python and OpenSCAD code to convert virtual items into physical ones. We created tools to enable not only Scratch nodes to be used, but also Unreal Engine 4’s Blueprints and Lucid Chart Flowchart symbols!

We have gone through many iterations to try and reduce print time. We have tried everything from printing as one big block, using Lego connectors and even printing each piece of text separately and mounting it on a perspex block. The latter approach is the one we have settled on, due to the current limits with 3D printing.

The students worked really hard on this project, with some of the year one Software Development and Games Programming students operating the new MakerBot Replicator.

As part of the year two students project work, they each wrote a website showing other people how to use the tools they have developed. We took the best parts from each website and added this to the ProjectViP.info website – this should be the one stop place to get more information on how to create physical objects from virtual ones.

This has been a great success, as Callum has finished the year with six distinctions and three merits! Well done to everyone involved, especially JISC for top rate support and helping us to approach the project in a structured way with clear objectives at each stage.

Shipley-college-team

Shipley-college-team

Finally, we are also looking at a spin off from the main project. We are placing individual letters, with text and Braille, on top of a Lego 2 x 1 base. Lego showed some interest in this and we are now looking at how to get this to market.

More details on the course at Shipley College can be found here

Leave a Reply

Your email address will not be published. Required fields are marked *