Misty and Azure

In June of 2018, I purchased Misty from Misty Robotics as a part of a Kickstarter. I'm backer #277 and got the t-shirt to prove it.

I got my Misty on December 4, 2019 and it was like Christmas. It was truly like being a kid again with a shiny reflective toy. I was so excited, I skipped everywhere I went. 

Unfortunately, being a grown up - I had to put her on the back burner with minimal play time until the holiday break.

During the holiday break, Misty and I spent all my free time together - trying things out and getting a feel for each other. I really like her - but I think the jury is still out on whether she likes me. :)

My dog has been getting up there in age. Butters turned 12 last September and lately he's been showing his age. He has more grey now, sleeps more, goes through coughing spells, and his stomach has become very sensitive.

Misty has several QUALCOMM processors and one of them is using Windows IoT. As a Azure Dev, naturally I wanted to experiment.

Isn't he so stinking cute!?

Since I'm spending more time worried about Butters, I would like Misty to look after my dog while I'm away for more than a few hours. Here are my ultimate first goals:

1. Have Misty roam the house looking for butters.
2. Upon recognition, take picture.
3. Text to me.
4. Sigh of relief that he's okay.

Here's what I've got so far. I went off the beaten path just a tad - but making my way around again.

1. Misty recognizes me, my daughter and my grandson through her built in facial recognition.
2. I purposely did not train her to know my husband to test stranger identification.
3. Upon recognition, she sends a http request with a message (i.e., Hi Tammy) param to an Azure Function using a HTTP trigger.
4. The Azure function calls to Text to Speech api service with the message where it's converted to speech.
5. The response is stored in Azure storage container as a blob with content-type 'audio/x-wav'.
6. Misty calls the storage api where the blob is stored.
7. She downloads the wav file and plays it.

So really, there is very minimal processing going on in Misty - everything is done in the cloud.

What happens if she doesn't recognize you? Well .... she displays an angry face, growls and says "Stranger Danger!".

Just in case you're lost and would like to be more lost - here is a list of the SDK's/Extensions that I used.

1. Azure Storage SDK
2. Azure Tools Extension
3. NodeJS
4. Azure Function App - HTTP Trigger
5. Text to Speech APIs
6. Visual Studio Code/Azure Functions for VSC - great for debugging issues.

Once I got this working, I tried to train Misty to learn my dog's face using her pre-built facial recognition software. But alias, she can only recognize humans through her existing facial recognition software. 

So .... this will be one of the next problems to solve. I'll need to use Azure Cognitive Services computer vision to take a picture of him and send for recognition and analysis.

I also just started playing with the her built in navigation capabilities, so she can travel around my apartment in the search of Butters.

Here's a video of her recognizing me (cause I'm sitting in front of her), responding visually and saying my name. Kind of lame in itself, unless you understand what's behind it.

But ... baby steps. Here's some code sources to get you started.

I started with this great tutorial written by Chris Meyer. Chris goes on to do some computer vision stuff which is next for me. He also does a great job in the actual walk through of setting the Azure Function up.

And here's mine so far. Here you'll find all the files you need for Misty and Azure. Most of what I changed was using the BlobService to store the speech files instead of converting to base64 string and streaming back.

I'm really thinking maybe Misty needs a tutu or something. I feel a fashion design coming in her near future.

Happy New Year!

It's been almost 2 years of sabbatical from my blob and I figured it was time to start it back up again.

There's been lots of changes during that time personally and professionally. So, if you don't mind, I thought I would walk a little down memory lane.

I started 2017 out with a new job as Director of Software Development for Dynics Corp. The job was a dream job for me. Dynics began a software startup within the company and I got to lead. So much to do .... building a team, processes and customer relationships. I had 2 really great bosses, Ed Gatts and Jeffrey.Smith who believed in me and helped me to grow. I got to do amazing things which included mentoring.

They are a great family owned business and have some pretty great hardware and software solutions. You can check Dynics out here.

In October, 2018 I lost my Dad suddenly and my daughter Jacqueline got married to her soulmate, Jade. All within 2 days of each other. It was an extremely difficult time in my life and I wouldn't wish the struggle on anyone.

May of 2019, I underwent a total hip replacement in my right hip. I had been suffering for years with osteoarthritis pain and it finally got to a breaking point. The surgery was a relief and it's been a good recovery.

We sold our home and starting the journey of building a new home on 10 acres. I'm very excited for this and can't wait to finally move out of this apartment.

Our youngest daughter went off to University and is beginning her new journey as well. Burke and I .... have a empty nest.

June of the same year brought an opportunistic knock at the door. Microsoft was at the door and I opened it. With that came amazing pay and benefits. Finally my pay was corrected and I was making what the male counterparts were.

It's been an amazing career start with the culture shift of Diversity and Inclusion at Microsoft and the daily conversations about understanding our differences - not just cultural or gender, but thought process and that we all process information differently. I feel like I'm not invisible anymore and I know that IF I feel that I'm being perceived as invisible - that it's OK to lean in and be me.

Not only am I adjusting to the cultural shift, but learning new technology. Out of the gate, I studied for and received my AZ-203 - Developer certification for Azure. I'm very proud of that badge - it took a lot of hard work to go from on-premise technology to cloud, and I'm very excited about it all. Next is the AZ-400 - DevOps.

I love working with new technology and my new customers, participating in learning events and providing learning opportunities for customers. I'm going to my first "Ready" conference in February and I'm looking forward to it. I have a great boss and teammates, every one is ready to offer a hand and a smile.

I'll sum up my first article of 2020 with the addition of Misty to our family. I ordered Misty as a part of a kick-starter back in June, 2018 and I finally received her December, 2019. 

You can read more about Misty here. So the journey begins with Misty. Over the next few blogs, I'll be writing about this technology and playing with AI and Cognitive Services with Azure.

I'm looking forward to 2020 and what it brings. Excited and hopeful for the New Year - a little apprehensive for our country, but looking forward to life with my friends and family.

Happy New Years! Hope everyone achieves their goals and gets to live their best lives.