AR-Based Tangible Programming Language
Yue Yuan --- Developer
Chen Xiang --- Developer
Yate Ge --- Researcher
US - China Hackathon Finalist
Coding, STEM, robotics; these have become buzzwords for what a 21st century education should be. How to make programming education into a more engaging experience and make it accessible to all kids, even the ones who cannot afford the huge budget for Programming Kit to have the equal chance to learn and make.
Our solution is developing an AR-based programming language, Where kids can implement the programming logic and build the scenario by jointing colorful cards. Also, immersive and interactive learning experience can be achieved with minimal budget.
In China, the changes in population patterns such as '4 + 2 + 1 family structure' and 'educational consumption upgrade' have made the overall demand for the early childhood education market increase rapidly, and more parents have a general demand for children's education.
The "nurturing care" and "game entertainment" have shifted to a more systematic and comprehensive "parent education" and "professional early education". Among them, the "children's programming education" market has received special attention.
Children's Programming Education Received Special Attention
Immersive Programming Learning Experience
Programming itself is a very complex subject, requiring a certain logic, abstraction, and mathematical foundation.
Compared with the traditional programming education based on graphical language learning, Creating an immersive experience is critical to programming learning. Learners guide characters through input code instructions, starting with zero threshold and increasing difficulty. The process contains both rich sensory experience and cognitive experience.
Augmented Reality Brings New Possibilities
New interaction methods
camera recognition + AR gestures
Combined with real-world spatial computing, rich in 3d content
Frame the Problem Statement
How to Use AR Technology for Programming Learning?
Concept 1 - Programming for Waterdrop Adventure
Our hero is a raindrop. Kids can give instructions (via simplified programming language) to help raindrop move and choose different channels until arrive at one floating island.
Three Floating Islands
IF Succeed ...
( I can do it! (ง •̀_•́)ง
Three Floating Islands
IF Fail ...
( I can't ... (ಥ﹏ಥ)
Water droplets at different states
Concept 2 - Multi-modular Tangible Programming
Create a personalized immersive AR experience with programming cards. Design your own AR games, AR stories, interactive experiences. And share and communicate with your family or friends.
AR + Programming Blocks
Logic programming through physical logic cards
Set user interaction, data, logic, etc.
Build scenes from space and object cards and connect to logical cards
Interact with virtual content through handheld devices (screen interactive controls, voice, accelerometers...)
Help the programming process with AR (using AR features to guide programming)
Create AR interactive experiments (sound, screen interaction, camera, accelerometer, etc.) with device features
Create multi-person AR interactive games together (multi-users create together and share experiences)
Interactive AI technology learning (encapsulation of AI algorithms, users experience machine learning and other ai technologies in ar)
Immersive scene package (users can build small physical experiments, simulate unmanned vehicles, robot control, shooting games, etc. according to different themes)
Create a personalized immersive AR experience with programming cards. Design your own AR games, AR stories, interactive experiences. And can share and communicate
Use different color blocks to represent different functional modules
Spatial physics experiment of multimodal interaction
Multiplayer interactive game
AI technology learning
Flow-based programming is designed with block-based programming language.
Spatial programming with physical cards.
Data flow control, calculation, transfer, and execution through splicing
Each card takes the generated texture for image recognition
Visual design, using icons, colors, interface arrows to help semantic understanding
The hexagonal design is suitable for both splicing and easy expansion.
Users create new modules through hand-drawn and software platforms