Incluvie: Actor Data Collection Ada Gok, Dana Hochman, Lucy Zhan

Transcription

1 Incluvie: Actor Data Collection Ada Gok, Dana Hochman, Lucy Zhan Figure 0. Our partner company: Incluvie. 1. Project Task Incluvie is a platform that promotes and celebrates diversity within Hollywood. The platform allows movie viewers to voice their opinions on a movie regarding diversity and displays movie diversity ratings. Our project with Incluvie is focused on actors and actresses. Our task is to create a method that extracts specific data parameters related to gender and ethnic backgrounds of the actor. With our actor dataset, we continue to create data visualizations to illustrate the gender and cultural diversity in Hollywood. Incluvie will use our code and data visualizations to display diversity in Hollywood. In addition, we will use the dataset to analyze any possible correlation between the gender and race of an actor and his/her popularity. Furthermore, Incluvie will be able to use the datasets/code to compile actor profiles for its users. Utilizing this data, users can easily determine the kind of gender and cultural diversity stance a movie holds by viewing a movie s cast and the actors gender/ethnic information. 2. Approach Our main method of data extraction involves taking in an input text file. The input text file is a list of actors full names listed line-by-line. Given that there is not yet a full list of actors Incluvie wants to generate data for, we start with grabbing data for the top 100 actors in Hollywood. The goal is to first write the code for data extraction of the top 100 and have it automated for all actors in the future. We chose two lists to generate two datasets: Top 100 of the Highest Grossing Stars of at the Domestic Box Office and Top 100 on IMDB Starmeter 2. These lists are compiled to a text file in the same format as previously stated. After the method reads each text file, it parses it line-byline. For each line (each actor), multiple APIs and Python packages are used to search the web for the actor s ethnicity and gender. In this project, we are focusing on the TMDB API, Wikipedia API, Ethnicelebs and Beautiful Soup package. TMDB API takes in an actor s name and does a search for his/her TMDB page. It proceeds to retrieve data from the actor s TMDB page. The specific data parameters we retrieve are gender, birth place and popularity score 3. Next, Wikipedia API retrieves actor s Wikipedia page and the page content in HTML format. Using the Beautiful Soup package, we parse the HTML content to search for the Early Life section of the Wikipedia page (we recognize that the Early Life section generally 1 "Highest Grossing Stars of 2018 at the Domestic Box Office." 2 "IMDb Top Star Meter." user generated list 3 Popularity score is a score determined by TMDB based on the actor s number of views for the day and scores from previous days.

2 contains an actor s cultural background). We proceed to extract that section and search for keywords ( ethnicity, ancestral, descent, ancestry, mother, father, mom, dad, parents). Within the same sentence the keywords appear, we search for ethnicity terms. We have a list of common ethnicities in a csv file that can be converted into a python list using ethnicities_list.py. This is our python code that specifically parses a csv file (in the format of our ethnicities.csv file) and converts it into a python list. We use this ethnicities list to do our search for ethnicity titles. The final ethnicity titles get saved as ethnicity list for the actor. Ethnicelebs is a website that allows users to submit specific actors and data regarding their ethnicities. A majority of our list of actors are found on this site. Although Ethnicelebs does not have an API, we can parse the HTML code of the actor s page using Beautiful Soup to find ethnicities of the actor. Ethnicelebs has a specific section that lists out the ethnicities of the actors therefore, we retrieve this section this string and clean it up to save only the ethnicity titles. We clean up the line by searching for the titles matching those in our ethnicities_list.py list. The algorithm is very similar to the one used for Wikipedia. Once we finish the data extracting process, we store it all in a data frame and export the data frame as an excel sheet. We then proceed to generate data visualization to showcase the gender and cultural distribution of Hollywood using the Python library Matplotlib, which provides graphing abilities. With this library, we were able to generate an assortment of bar charts, pie charts and histograms (see the resulting figures in the graph appendix). In addition to graph visualizations, we want to analyze any correlations between gender and race with the popularity of actor (based on TMDB popularity score). We plan to find the correlation by testing through decision trees. We create an algorithm using decision trees that classifies an actor's popularity based on the actor's race and gender. The decision tree takes in a data frame of gender and race for all our actors (represented as binary variables {0,1}). Using this data, the tree predicts whether an actor s popularity score falls within the range of low, medium, or high. We split the actor s popularity into these three categories by splitting the popularity range (difference between lowest and highest score) into thirds. The decision tree visual (see graph appendix Figure 27) follows the path our algorithm takes in computing the popularity score. The decision tree trains 80% of the data and tests on 20%. The goal is to see how accurately a decision tree can classify an actor s popularity score based solely on race and gender. 3. Dataset and Metric For this project we have two final excel files of data: topactors_data and topimdb_data. The file alldata.xlsx contains the combined data of both. Each file represents data for a different list of corresponding actors (Top 100 of the Highest Grossing Stars of 2018 at the Domestic Box Office 4 as topactors.txt and Top 100 on IMDB Starmeter 5 as topimdb.txt, respectively) 6. A sample of the excel sheets are attached at the graph appendix (refer to figure 1-4). 4 "Highest Grossing Stars of 2018 at the Domestic Box Office." 5 "IMDb Top Star Meter." user generated list 6 From our analysis, IMDB star meter seems to focus more on the recently popular stars

3 The final excel sheets not only contain the data parameters that were extracted (gender, birth place, ethnicity list from Wikipedia, ethnicity list from Ethnicelebs, and popularity score), but also additional columns: percent similar, percent difference, race (sourced) and race (gut check). Percent similar/different are parameters for cross-checking the error between the ethnicities list generated from Wikipedia and Ethnicelebs. We calculate the value for percent similar by finding the number of ethnicity titles that match in both the Wikipedia and Ethnicelebs list and dividing it by the number of total unique ethnicity titles in both lists (for example, if Wikipedia and Ethnicelebs match on 3 out of 4 total ethnicities for one actor, this would be 3/4 = 75% similarity). The percent different parameter is 100% - percent similarity. This parameter represents how many of the ethnicity titles are not found in both lists. Race (sourced) represents the actor s race based on the actors ethnicities on Wikipedia and Ethnicelebs. Race (gut check) represents the actor s race determined by guessing 7. The reason we incorporate both race (sourced) and race (gut check) is to cross-check the data to ensure accuracy. Using the data, we generate multiple graphs relating to gender, popularity, ethnicity and race (e.g. gender histogram, popularity distribution histogram, birth country distribution histogram, etc.), which can be found in the graph appendix (refer to figure 5-26). These visualizations display the data to allow the viewer to easily draw conclusions. In order to also utilize our data, we created a decision tree to classify an actor s popularity based on gender and race. 4. Disclaimer/Assumptions Regarding this project and the methods, we want to make a few disclaimers and clarify some assumptions that were made. First, the actor dataset we are providing is made of data grabbed from three sites: TMDB, Wikipedia, and Ethnicelebs. TMDB is a database similar to that of IMDB and so the assumption is that the data is relatively accurate. Wikipedia and Ethniceleb pages consist of data that is edited and compiled by users, so the accuracy and reliability is questionable. However, we assume Wikipedia and Ethnicelebs users cross-check this data as they make edits and thus, it can be reliable. Furthermore, extracting data from both Wikipedia and Ethnicelebs helps validate the ethnicities. Additionally, our percent similarity/different parameters provide a value that can be a status of reliability. Second, we want to make the claim that not all data is available online, so there is missing data in our datasets. Third, regarding the data we added manually (e.g. race -- sourced and gut check), we labeled actors as Latino(a). We are aware that people of Hispanic origin do not classify Latino(a) as their race, but it was a specific factor we found necessary to clarify. Fourth, in reference to our ethnicities list generated from Wikipedia and Ethnicelebs, we are aware that certain titles (e.g. American, African American, and European ) are not ethnicity terms. However, Wikipedia and 7 We understand some bias could be involved in this process.

4 Ethnicelebs both list these as ethnicities and we felt it was important to include them so as to not lose data. This disclaimer is mirrored for the figures and figures in the graph appendix. 5. Results Referring to our gender bar graphs (figure 5 and figure 15 in the graph appendix), we can make specific conclusions regarding the gender representation in Hollywood. Figure 5 is the gender split amongst the Highest Grossing Stars of 2018 at the Domestic Box Office 8 and figure 16 is the gender split amongst the top Top 100 on IMDB Starmeter 9 (which we observe to be the current actors rising in fame ). We conclude that the highestgrossing stars have a 60:40 male to female ratio (figure 5), but the rising stars have a lower male to female ratio (figure 15). Furthermore, we analyze graphs regarding ethnicity distribution amongst all actors. Figure 10 and 11 (refer to graph appendix) are examples of the ethnicity distribution among the Highest Grossing Stars of 2018 at the Domestic Box Office 10. Observing these graphs, we can conclude that the majority of our actors are of European or American descent. Likewise, we analyze nationality distribution among the actors. Figure 9 of the graph appendix shows a nationality distribution stack bar chart by gender. The bar chart does not show any significant takeaways regarding gender and cultural representation in Hollywood. Most of the actors are from the US and within the actors with US nationality, majority are also male. Additionally, we create some graphs regarding race (sourced) shown in figure 15 and 26 of the graph appendix. These graphs continue to show that the white race overthrows all other races in terms of representation in Hollywood (like the ethnicity graphs). To visually demonstrate where actors are born, we created a geographical heat map to showcase where most actors in Hollywood come from. This is also a nice interactive addition for the Incluvie website (refer to figure 28 in graph appendix). For the decision tree, we run all 200 data points over it. The resulting accuracy is very high: 90% accuracy. 6. Conclusion As we wrap up our project for Incluvie, we want to reflect on the project, the challenges and the learning experience. Gathering data was one of the bigger challenges. It took time to find APIs that retrieved data we were looking for and figuring out how to parse through information on webpages. This was especially true with parsing through Wikipedia, where we had to continually expand our keywords list in order to find the ethnicities mentioned within paragraphs of text. However, through our data extracting, we learned how to effectively utilize APIs and the best methods for parsing through webpages. One concern that still remains is accuracy, as our data is not complete and is reliant on websites with user inputs (ex. Wikipedia and Ethnicelebs). 8 Highest Grossing Stars of 2018 at the Domestic Box Office." 9 "IMDb Top Star Meter." user generated list 10 Highest Grossing Stars of 2018 at the Domestic Box Office."

5 Overall, from this project, we have some conclusions to make about representation in Hollywood. We found that the gender representation is definitely unbalanced. When we analyze the data for the top 100 grossing actors, we get a 60:40 ratio of male to female. However, we realize that the actors rising in popularity (top IMDB starmeter) has a gender ratio that leans towards females. In one way, this can be viewed as a good sign, as it shows that there is improvement in gender representation in Hollywood. However, it can also be viewed negatively because despite the growing popularity of female actors, they are still being paid less. In addition, we can make conclusions about the cultural representation. As showcased by both ethnicity and race graphs, the White race conquers the Hollywood industry. From the decision tree we were able to classify actor popularity score ranges with a 90% accuracy. A high accuracy generally infers a high correlation -- thus, we can conclude that there is a high correlation between popularity, and the actors race and gender. However, we must recognize that we have a small data set and most of our actors have scores in the low range. The fact that a majority of our top actors have low popularity scores also casts some doubt on the TMDB popularity score. It is somewhat unclear as to how it is calculated, and there is no rationale as to why the range of popularity scores among certain actors varies so widely. A metric such as the F1 score could pinpoint the balance between precision and recall of the results. This would tell us how accurately the model identifies and returns all relevant popularity scores of the actor dataset. Perhaps it would be best to use a different measurement of popularity for future analysis. 7. Future Work While working on this project, we believe there are parts that can be improved in the future. As we ran the code, we believe there are parts that could be refactored to improve efficiency. The current code for our data extraction is not very time efficient. This could be the cause of parsing through long HTML codes. The run time does not affect our current work as the dataset only needs to be generated once. If we want to run it in the future for a larger set of actors and do so more regularly, it will be very time consuming. Therefore, it is necessary to optimize the run time. In addition, we wish to fix the code for data extraction and make it more dynamic. As of now, the code takes in a text file of a list of actor names. For future work, we want to find a reliable list of top 100 actors that is automatically updated daily/weekly. As the list updates, it would be ideal if we could automatically extract new data or use old data to update the graph visualizations for the 100 actors in Hollywood. Furthermore, we wish to find a more reliable source to gather a list of ethnicities for actors. Wikipedia and Ethnicelebs are both based on user input, which is not fully reliable. A more reliable list would not only increase the accuracy of our data, but also increase the amount of data, as we currently have some missing ethnicities for some actors. References 1. "Highest Grossing Stars of 2018 at the Domestic Box Office." The Numbers - Where Data and Movies Meet. Accessed December 03,

6 2. "IMDb Top Star Meter." IMDb. Accessed December 03,

7 Graph Appendix Figure 1. Data frame of Top 100 Highest Grossing Actors of 2018

8 Figure 2. Topactors_data.xlsx snippet

9 Figure 3. Data frame of Top 100 IMDB Starmeter Actor

10 Figure 4. TopIMDB_data.xlsx snippet

11 Figure 5. Gender Bar Chart for Top 100 Highest Grossing Actors of 2018

12 Figure 6. Popularity Distribution Histogram for Top 100 Highest Grossing Actors of 2018

13 Figure 7. Nationality Bar Chart for Top 100 Highest Grossing Actors of 2018

14 Figure 8. Nationality Distribution Pie Chart for Top 100 Highest Grossing Actors of 2018

15 Figure 9. Nationality Stack Bar Chart Split by Gender for Top 100 Highest Grossing Actors of 2018

16 Figure 10. Ethnicity Bar Chart for Top 100 Highest Grossing Actors of 2018

17 Figure 11. Ethnicity Distribution Pie Chart for Top 100 Highest Grossing Actors of 2018

18 Figure 12. Ethnicity Stacked Bar Chart by Gender for Top 100 Highest Grossing Actors of 2018

19 Figure 13. Nationality (U.S. States) Bar Chart for Top 100 Highest Grossing Actors of 2018

20 Figure 14. Nationality (U.S. States) Distribution Pie Chart for Top 100 Highest Grossing Actors of 2018

21 Figure 15. Race Bar Chart for Top 100 Highest Grossing Actors of 2018

22 Figure 16. Gender Bar Chart for Top 100 IMDB Starmeter Actors

23 Figure 17. Popularity Distribution Histogram for Top 100 IMDB Starmeter Actors

24 Figure 18. Nationality Bar Chart for Top 100 IMDB Starmeter Actors

25 Figure 19. Nationality Distribution Pie Chart for Top 100 IMDB Starmeter Actors

26 Figure 20. Nationality Stack Bar Chart Split by Gender for Top 100 IMDB Starmeter Actors

27 Figure 21. Ethnicity Bar Chart for Top 100 IMDB Starmeter Actors

28 Figure 22. Ethnicity Distribution Pie Chart for Top 100 IMDB Starmeter Actors

29 Figure 23. Ethnicity Stacked Bar Chart by Gender for Top 100 IMDB Starmeter Actors

30 Figure 24. Nationality (U.S. States) Bar Chart for Top 100 IMDB Starmeter Actors

31 Figure 25. Nationality (U.S. States) Distribution Pie Chart for Top 100 IMDB Starmeter Actors

32 Figure 26. Race Bar Chart for Top 100 IMDB Starmeter Actors

33 Figure 27. Decision Tree in regards to popularity

34 Figure 28. Snippet of Heat Map of the Birth Place of all data points