Tools & Technologies used in projects:
Python, Solr, Elastic Search, Kibana, OpenCV, TensorFlow, PyTorch, Raspberry Pi, Alexa, DialogFlow (previously API.ai), Tableau, Google Cloud, AWS
Presented AI / ML demos to senior executives in one of the largest sports television channels on the following topics:
- Sports Activity Recognition: Classifies images by recognizing sports activities (e.g. swimming, snowboarding, ice skating, skiing etc) using Transfer Learning.
- Video Tagging & Meta-Data Extraction: Extracts meta-data from sports videos using techniques like Optical Character Recognition (OCR), Speech-to-Text and Face Recognition, to identify players, teams, sports, tournament and venue details.
- Contextual Advertising: Displays product ads for sports merchandise and memorabilia based on which teams or players are seen in sports videos.
Mentored a team of 7-8 interns from BITS Pilani on the following AI / ML Projects:
- Product Image Search & Visual Similarity: A product search-engine which takes an input image and searches the product catalog for visually similar items using Deep Learning library, PyTorch. The algorithm is applied to various product categories: clothing (shirts, skirts, jackets, dresses etc), home decor (curtains, bedsheets, lamps, carpets) and accessories (shoes, watches, jewelry, handbags etc).
- Contextual Advertising: Recognizes objects in the given video stream, and searches product catalog to find similar items. Product ads for similar items are embedded using video overlay. This concept is applied to Fashion Show videos, to find dresses and clothes similar to those worn by fashion models on the screen.
- Sports Video Analytics: Automatically generates highlights for sports videos using audio-visual features that capture the excitement and action in the game. Also, face recognition is used on close-up shots to automatically display career highlights or brief bio for players on the screen.
- Entity-Relation Graph: Displays relations between sports players and the teams they play for, as well as, between the teams and the leagues they are part of, using the text extracted from Wikipedia. Stanford’s Core-NLP library is used for parsing text to find entity-relations, and D3.JS for creating visualizations.
- Smart Cities: Analyzes video streams from traffic cameras to count the vehicle flow (number of cars, trucks, bikes and buses passing) on the street. The same algorithm is also extended to create Smart Parking app, which captures an image of a parking lot, and creates a map of empty and full parking lots, to guide users to the nearest empty parking lot.
Projects on Raspberry Pi, Alexa & Wearable devices –
- Media Player: Car Infotainment System built on Raspberry Pi
- SpiCam: Video Surveillance Camera for Smart Homes
- Smart Fridge: Recognizes Items in the fridge using the AWS Rekognition library
- Programmable Jewelry: Smart Jewelry using Near Field Communication (NFC) technology
- Fairytell Bot: Alexa skill to read bedtime stories from children’s classics
These projects can be found on Hackster at – https://www.hackster.io/pamruta.
Guided a team of 6-8 junior data scientists on the following AI / ML projects:
- Smart Kiosk: solution developed for retail customers. Customers scan products at Smart Kiosk to see product information like its features and specifications, nutritional facts and recipes for food items, and related products for cross-selling. Smart Kiosk uses Computer Vision to recognize fresh produce like fruits and vegetables, and Optical Character Recognition (OCR) to read product labels, brand names and logos.
- Video Highlights Generation: The algorithm uses audio signal processing to detect loud cheering, clapping and applause for automatically generating highlights for sports videos.
- Damage Detection: This deep-learning based image classifier uses TensorFlow model to identify signs of visible damage like scratches or cracks on mobile phone screens, car windshields and glass windows to speed up the process of insurance claims processing.
- Audio Classification: Classifies ambiance sounds as laughing, crying, screaming, shouting, clapping and cheering using MFCC (Mel Frequency Cepstral Coefficients).
- Video Tagger: Identifies topics from Tedx Talks, Online Course Videos, Lectures and Tutorials by running Speech Recognition on the audio track.
- Health Bot: A conversational chat interface to search nearby doctors and hospitals by specialty areas like cardiologists, dentists, pediatricians etc.
- Video Analytics: Face Recognition, Pedestrian Detection, Vehicle Number Plate Recognition for surveillance cameras.
The AI / Cognitive team built all the above demos within 6 months, earning the Innovation Team Award in Dec 2017.
Other Responsibilities taken at Happiest Minds:
- Member of CTO Technology Council
- Member of Diversity and Inclusion Council
- Submitted a proposal on AI Chat-Bots that fetched the revenue of US $100K. This chat-bot was built on Azure stack using the Microsoft Bot Framework and automates operations for an IT support help-desk.
- Prepared a complete road-map for AI / Cognitive Group at Happiest Minds, presenting solutions and offerings on Audio, Video, Image and Edge Analytics.
- Won Team Excellence Awards on various customer projects
Experience with Solr, Elastic-Search and Kibana:
- Built a Kibana Dashboard to visualize crowd-funded projects on Kick Starter. Projects can be visualized by categories, themes, launch date and funding received.
- Created a culinary search-engine to search thousands of online recipes based on the given ingredients (e.g. rice, lentils, potatoes, spinach, corn, mushroom) or category (e.g. sandwich, soup, cake, breakfast, brunch, vegetarian). Sample search-queries include: “apple pie”, “carrot cake”, “mushroom soup”, “potato salad” etc.
Projects done in MakeMyTrip –
- Built a destination-search engine by mining articles on Wiki-Travel, to suggest top domestic and international cities for a selected activity (e.g. scuba diving, ice skating, cross-country skiing, trekking, horse riding etc) or theme (e.g. wild life safari, hill station, beach resort, amusement parks, world heritage sites).
- Built a hotel-search engine by mining reviews from Trip Advisor. Hotels can be searched near specific points-of-interests (POIs) like metro station, airport, popular landmarks, local attractions and neighborhoods, or by specialty services like Italian Dining, Infinity Pool, Private Beach etc.
- Performed sentiment analysis on text-snippets (phrases) extracted from online hotel reviews to score and rank hotels on various dimensions like location, quality of food, service, cleanliness and amenities.
- Trained a word2vec model on Trip Advisor reviews to automatically discover concepts related to travel domain from text. Also created visualizations by projecting word vectors in 3D space using Embedding Projector visualization tool from Google.
Projects done in Persistent Systems Ltd –
- Automatically generated skill-profiles for technical support engineers by mining unstructured text-content from email communications. The algorithm retrieves a ranked list of experts in the given skill-areas.
- Built a demo prototype for a client that works in medical and health-care domain by analyzing the content posted on social media. The project identifies trending topics, user activities, as well as, the sentiments in user posts, to study the effects on product sales and revenue.
- Performed predictive analysis for a client that offers bike rental service in the Bay Area, by computing correlations between bikes rented and weather parameters like temperature, wind speed, humidity etc. on a given day.
- Developed forecasting models for predicting stock prices for Fortune 500 companies based on the historical data for the past 10 years using Time-Series Analysis.
- Implemented brand-clustering algorithm to identify companies that work in the same industry sectors or offer similar products and services, by extracting features from online news text using word2vec utility.
- Mentored a team of college interns from College of Engineering, Pune (COEP) on “Box-Office Predictions” project, to estimate the box-office revenue for movies, based on parameters like reviews & ratings, production budget, studio, cast and genres.
Earned “You Made a Difference” award in Oct 2015 and March 2016 for contributions made on the above projects.
Projects at University of Pittsburgh:
- Proposed a novel technique for detecting and analyzing humor in comedy television show FRIENDS. This research was published in one of the top international conferences in Natural Language Processing area (EMNLP) held in Sydney, Australia in 2006. The paper analyzed dialog transcripts and audio recordings in FRIENDS TV-show for automatic humor detection.
- Proposed a Machine Learning framework for analyzing coherence in spoken conversations. The algorithm tries to distinguish random incoherent conversations from natural coherent dialogs with over 85% accuracy. This research was published in FLAIRS 2008 conference held in Florida.
- Implemented Text Mining and Information Extraction algorithms for automatically building a large-scale relational database for movie actors and pop-singers by mining web pages and biographies on Wikipedia. This project was carried out at SONY Corporation, Japan during summer 2008 internship program.
- Explored data mining techniques to automatically categorize product items based on the similarity of their features and product descriptions. This project was done at Amazon.com, Seattle during summer 2005 internship program. The project utilized unsupervised clustering algorithms to automatically build a product taxonomy by grouping similar products.
- Implemented a language semantics demo for analyzing similarities and relations between entities from natural language texts. This project was done at the Information Sciences Institute (ISI) of University of Southern California (USC) during summer 2007 internship.
Developed classification algorithms for predicting the email reply order for Automatic Email Prioritization. The project analyzed user behavior and inter-personal relationships among users, along with the features extracted from emails to predict the email-reply order for prioritization.
Worked on a Contextual Advertising project for a start-up company, Social Extract. The project analyzes text content on Twitter to identify twitter users and relevant tweets for advertising and marketing campaigns.
Projects at University of Minnesota:
- Developed an open source software package SenseClusters for Unsupervised Word Sense Discrimination task. This project was funded by the National Science Foundation (NSF) research grant and was completed as part of the Master’s Thesis at University of Minnesota.
- Worked with collaborators at Mayo Clinic in Rochester, Minnesota on bioinformatics project to develop unsupervised learning methods for resolving ambiguities in biomedical texts.
Won 2nd Prize for Best Paper in Artificial Intelligence & Fuzzy Logic in technical/research paper presentation competition organized by Pune Institute of Computer Technology (PICT), India in association with IEEE. The paper presented a prototype model for a language understanding system using parsing and logical inference.
— Last Updated in Sep 2018