Create your own JARVIS

With AIR, AS3 and PHP

Hi there. I hope you are well.

In this new tutorial, we will see how to create a “Siri like” mobile application compatible with Android and iOS.

This application will be coded with AS3 and compiled using Adobe AIR. The remote program (Program-O in this example) will be installed on your own web server.

If you want to see what can be achieved by programs, you can take a look on your favorite search engine for something like “AIML program” or “nlp machine learning”.

We will use Starling/Feathers as frameworks, and ezSTT/ezSpeech as ANEs to handle natural speaking functions.

So… It will work same on Android, iOS by the holy power of AIR, AS3 and ProgramO.

Introduction

Natural speaking concepts are interresting only if the machine can answer something cool when we speak to it. This one can be followed by an action (but this will be the subject of another post).

In fact, what we gonna build today is a modest Siri (or Google Agent) like system.

We will talk to it. And, that’s the magic of programs, it will give answers.

Don’t put the blame on me if your bot don’t answer what you want it to. You’re the boss.

YOU ARE THE ONLY MASTER OF YOUR OWN “JARVIS”!!!

Before we start this first part, we need to download some tools (if you read this post, you use those tools).

Starling

https://gamua.com/starling/

Feathers

https://feathersui.com

AIR SDK & Compiler

https://www.adobe.com/devnet/air/air-sdk-download.html

Starling / Feathers SDK and AIR SDK & Compiler All In One

To simplify this part, you can download Feathers SDK Manager.

https://feathersui.com/sdk/download/

Spinner (from Marpies)

– I really love your work, it saves me lot of working days –

https://github.com/marpies/material-design-spinner

Program-O

https://program-o.com/v2/download

ezSTT

https://fabricemontfort.com/product/ezstt-ane-air-native-extension/

ezSpeech

https://fabricemontfort.com/product/ezspeech-ane-air-native-extension/

Everything is on your computer ? Ok! It seems that you are now ready to start to build your bot.

The link to the entire source code and a video demo are available at the end of this post.

Installing Program-O

The “intelligence” of this system is not designed by me. I’ll only show you a small part of what can be achieved with a conversationnal agent. Maybe this code base will be useful for your future projects.

AS3 and AIR are not required to achieve this, this can be ported to every language that can touch Speech APIs.

Let’s start with this little exercise…

First, uncompress the Program-O archive to your computer.

Open you favorite FTP Client.

Upload the content of Program-O archive to your web server in “program-o” folder. If you want to choose a different name, YOU can, and please: “remember this one ’til the end”).

Create a new database on your server (this may be irrelevant if you want to use the same database for all your online projects – I personnaly prefer to have a private database for each project – maybe something like I love to put every idea in its private box… Yeah I’m a nerd).

Create a new database (utf8_unicode_ci)

Open your favorite browser and point to Program-O install folder:

https://your_webserver/program-o/install/ (for me it was https://localhost/program-o/install/).

Check (twice) that everything is OK and click “continue”.

Check twice that everything is OK and click “continue”

On the next screen you’ll have to enter some informations. Your name, email address, the name of your bot, response format (I use JSON for this tutorial, but you can use HTML or XML for your own use), and most important: MySQL settings and debug informations.

One time again, check twice your informations to make Program-O working properly. Then, click “save”.

Enter your informations, verify twice your inputs and then click “save”

Log into the Admin page of Program-O with your credentials. And go to “Bot Personnality”, then fill up the blank fields with your own bot informations. After you add your bot infos in the fields, click on “Add bot infos”.

Add infos to your bot personnality

Test your bot

Inside your Program-O Admin page, go to “Test your bot” and start with a simple “Hi”. If everything is OK, it should answer with “I do not know this. Teach me please!”.

If you get some error on this, please check again the previous steps. If you get some weird error, Program-O team is reactive and will help you to get it working. AND it’s an open source language, you can fork and contribute.

Go to https://github.com/Program-O/Program-O/issues or https://program-o.com/v2/contact

If you get no error, you are now ready to teach some basics to your bot.

Add simple interactions to your bot

For this tutorial, we only add 2 files. The first one (greetings.aiml) lists the ways a user can say “hello”. The second one (username.aiml) will handle the ways a user can talk about is name with the bot.

In both files, we have an enumeration of possible user inputs and the logic (srai) to compose an answer.

For a complete list of tags of AIML markup language, you can take a look at this cool site (and explore further):

https://www.tutorialspoint.com/aiml/aiml_basic_tags.htm

Let’s go!

greetings.aiml

This file is a very simple list of patterns which calls the same srai template.

username.aiml

This file is a little experiment to handle multiple ways to store and retrieve the username. It can be modified to fit your needs or, ever better, upgraded to be more efficient.

If you want to re-invent ALICE, you can get an english version here (that’s a base):

https://github.com/fastcoding/aiml-en-us-foundation-alice.v1-6

There’s tons of bot personnalities and complete sets of AIML files on the web (in number of langs), you can search, download and install these files freely in most case.

From here, you can play with your bot on a few words. Let’s see how to “really” talk with it using natural language.

Creating the interface on mobile

I use OSX as an OS and IDEA IntelliJ as a IDE. Please feel free to adapt this to your own environnement.

Create a new project. It’s a Flash/AIR Project (oops… Animate… oops… AS3… rhhhhh…. I’m so sad to cannot name it… like Voldemort…)…

Adobe… If you don’t know what to do with AIR, let’s talk… I know my fellows want to use it on PS3/4, XBox and Wii… We have a Mustang GT and we only drive on country roads.

For this tutorial I use Starling/Feathers MXML. First cause I like this kind of cool abstract. And, finally, cause I love this kind of cool abstract. Here’s the very simple interface. A single screen with two images, three labels and a touch event. To speak, touch the screen of your mobile device, the bot will answer.

For basic explainations, see the comments.

As always, you can download the full project with libs, assets, config files (for IDEA IntelliJ) on GitHub.

I think everything is online… It’s late for me…

https://github.com/Fabrice-Montfort/JARVIS-AS3-AIR-Voice-Interface-for-AIML-Bot-

I really hope this tutorial was useful for you and show you some new ways to do programs.

Feel free to send me a message, follow me on Twitter, Facebook or something else.

And as far that I know (and test it), it can be weird…

 

Do you want something even more cool ?

Stay tuned, I will publish in a few days (I promise) other tutorials to let you interact with virtual words (typically your game) or event better with the real world (IoT).

And don’t forget… AS3 and AIR rocks !!!!