Tanisha Final Checkpoint Plan

[tanishapatil1234]

Objective

The objective of this project is to implement an AI model that customizes a game character based on a picture taken by a camera. The model will analyze the picture to determine the user's skin color, shirt color, hair color, and hair length, and use this information to create a customized sprite/character.

Tasks

• Develop and integrate a camera feature to capture the user's picture.
• Create an AI model to analyze the picture and extract details about skin color, shirt color, hair color, and hair length.
• Design and create character sprites that can be customized based on the extracted features.
• Integrate the AI model with the character customization system.

The Plan

Requirements

• A camera feature to capture user images.
• An AI model to analyze images and extract relevant features.
• Customizable character sprites based on extracted features.

Camera Feature

• Develop a camera interface that can capture images from the user's device.
• Ensure the camera captures images in a resolution suitable for analysis.

AI Model

• Define the dataset and gather images for training the AI model.
• Train the AI model to recognize skin color, shirt color, hair color, and hair length.
• Validate the AI model with a test dataset to ensure accuracy.
• Deploy the AI model to integrate with the camera feature.

Character Sprites

• Design base character sprites with customizable attributes.
• Create variations for different skin colors, shirt colors, hair colors, and hair lengths.
• Implement a system to dynamically change sprite attributes based on AI model output.

Integration

• Link the camera feature to the AI model for real-time analysis.
• Connect the AI model output to the character customization system.
• Ensure seamless integration between capturing an image, analyzing it, and customizing the character.

Schedule

Monday:

• Share the project plan with the team.
• Begin ideation for camera interface and character sprite designs.

Tuesday:

• Develop the camera feature to capture images.
• Start gathering and preparing the dataset for AI model training.

Wednesday:

• Train the AI model using the prepared dataset.
• Design base character sprites and their customizable attributes.

Thursday:

• Validate the AI model with test data.
• Create variations of character sprites based on potential AI outputs.

Friday:

• Integrate the camera feature with the AI model.
• Connect the AI model to the character customization system.
• Conduct end-to-end testing to ensure the system works smoothly.

Milestones

• Camera feature implemented and tested (Due: Tuesday)
• AI model trained and validated (Due: Wednesday)
• Character sprites designed and variations created (Due: Thursday)
• Full system integration and testing completed (Due: Friday)

Next Wednesday's Presentation Plan

Presentation Outline

1. Project Objective

   • explain the goal of the project: to use an AI model to analyze a picture taken by a camera and customize a game character based on the user's skin color, shirt color, hair color, and hair length.

2. Camera Feature

   • demo the camera interface that captures the user's image.
   • try to explain the technical aspects of how the camera captures and processes the image for analysis.

3. AI Model

   • Describe the AI model's purpose and the process of training it.
   • Show the dataset used for training, highlighting the types of images and features it learns to recognize.
   • Present the validation results to demonstrate the model's accuracy in identifying skin color, shirt color, hair color, and hair length.

4. Character Sprites

   • Showcase the base character sprites and their customizable attributes.
   • Display the different variations created for each feature (skin color, shirt color, hair color, and hair length).

5. Integration

   • Explain how the camera feature and AI model are integrated.
   • Demonstrate the workflow from capturing an image to customizing the character.
   • Highlight the seamless transition and real-time customization.

6. Challenges and Solutions

   • Discuss any challenges faced during the development process.
   • Describe the solutions implemented to overcome these challenges.

Demo

• using different demo images

[tanishapatil1234]

Updates :

1. MODEL: I have created and trained the model. This model has been modified to predict based on age and gender. After reconsiderations with my group, we determined that by avoiding race we can avoid offending people if the model is inaccurate. Here is a tangible testing file in which I first created the model from the UTK kaggle dataset :

• Because the model is in python, to be usable on the browser I had to save as a python script -> save as two seperate files for .weights.hf and .hf . Then I have to use tf js to call in the html camera page.
  ** edit : from running the model my laptop is having trouble with syncing my changes. I can show this notebook live but for now here is a screenshot :

2. SPRITES : I have updated the sprites to disregard race as seen in the original plan. I have uploaded them to the profile picture section of the repo. I have also attributed each to a custom age range.

3. CAMERA : I have updated the camera to svae picture to local storage. Before it was just live predictions but the input of the model is a still so I have to eradicate the live feature. Maybe I can add a feature where the image used in the model itself is shown on the screen with some face trackers.

Essentially

I faced a LOT of errors this past week. I spent the majority of the week debugging and creating the model itself. Now I am facing deployment problems! Some errors : mae incompatibility , keras and tf version mismatches, and an incorrect conversion to js. I have it working on ipynb and webcam as an alternative, but today and tomorrow I am really going to work on getting the flask server to deploy. Or worst case scenario, I can use the webcam as an api, and send the results to codemaxxer frontend in order to then select the appropriate character.