I’m writing this from the perspective of a data scientist at an app development company. The company designs various apps and is interested in improving their average ratings in the Google Play app store. I found a nice dataset to work out of and wanted to answer this question:
Does changing the price of an app affect the average rating the app is given?
Either outcome to this question will give actionable results. Here is a snapshot of the data I used:
10,841 rows, 13 columns, quite a bit of missing entries especially in the ‘Rating’ column, and most of the data is stored as objects despite being numeric so cleaning is required. Since Rating is an outcome of my question, I would impute data in for these missing results by finding other rows close to the rows with missing values and find an average as a best guess.
I engineered an additional 29 features to help explore the data. Some were categorical, some were transformations of the features to try to get the distribution closer to a gaussian to help find correlation. In order to answer my question, the two main features I want to examine are Price and Rating.
The transformations don’t always look “nicer”, but they help compress eliminate the skew in order to help find correlation with other features.
Price is a special case because most of the data is at 0 since most apps are free. Not very helpful.
Looking just at apps that were paid. The population we are interested in is just paid apps, since we’re considering raising our price.
If we want a better rating, should we make our app free? Here’s a boxplot looking at the rating distribution of free and paid apps. According to this, no, we should not. Actually, paid apps are rated higher! We could explore this with a different experiment.
Instead, I want to explore just the paid apps, and see if adjusting the price will affect the average rating the apps receives. This is a scatterplot of our transformed variables for price and rating. Visually, this appears very close to the weakest correlation I can imagine. Before jumping into an experiment, I want to explore different groups in the data first.
Our population broken down into groups by number of installs: Strongest correlation coefficient is -0.32 for apps with 100 installs.
Content Rating: Strongest is -0.2 at Teen rated apps.
Number of Words in Title: Thought I’d check this out, strongest is for 1 word titles with a correlation coefficient of -0.36.
I also investigated category, number of reviews, and how recently the app was last updated. The highest correlation I could find between Rating and Price for all those gorups was -.38 for apps that were updated a “medium” long time ago. All of these correlations are very weak, and I felt sufficiently safe to explore the the following hypothesis:
Increasing the price of the app will NOT change the average rating.
In order to test this, I propose the following experiment: An app we have developed has been on the market for a year now, and it costs 1.99 to install. We should raise the price to 3.99 for 3 months and compare the average rating it receives during those 3 months to the average rating it received in the last 3 months. During this time we will not update the app in any way to remove potential biases.
Each data point is the average rating for an app in the Google Play store. The data was transformed exponentially to try to normalize the data from its left skew, and only apps that were priced were included. A test for normality in SciPy resulted in a p-value of 0.503. High p-values indicate this sample comes from a normal distribution so we can guardedly work under that assumption.
NormaltestResult(statistic=1.3724099024430614, pvalue=0.5034831912361784)
To determine if it has significantly lowered in rating, I will do a ‘Independent sample t-test’ with α = 0.05 :
Null hypothesis: Increasing the app will not change the average rating
Alternative hypothesis: Increasing the price will change the average rating
Given that priced data after being exponentially transformed is normally distributed, we will be able to perform a t-test to compare the means of our ratings for the two 3-month windows. We will have to verify that our test and control are normally distributed once we have collected the data. If they are not, then we will use a non-parametric test.
One possible bias that we can verify immediately is average rating by month. I will analyze the average rating by month for our app to see if there’s any difference in means. If so, we would have to run the experiment 9 months later to compare to the same 3 months from the control.
Once we compare the means using an independent sample t-test, there are two possible outcomes:
- Our p-value is greater than α = 0.05 and we will fail to reject our null hypothesis. Increasing the price did not change our average rating. I will recommend that we increase the price of our app to 3.99 since it will not affect our average rating in a negative way.
- Our p-value is less than α =0.05 and we can reject our null hypothesis and conclude that raising the price will negatively impact our average rating. I will recommend that we keep the price the same, or consider pursuing further research into lowering the price of the app.
Limitations:
- We cannot control for year. If year somehow biases the average rating then it would invalidate our results.
- We are also only performing this experiment on 1 app, and so we can only make soft conclusions for expanding these results to other apps. It’s possible a different category of app would have different results in this experiment.
- We can’t draw conclusions on more dramatic increases in price. Results from a $2 increase might not be the same as from a $20 increase. I would feel uncomfortable advising a more intense price hike.
