đź“Ł Chancing Algorithm Updates - May 2021
TL;DR: We recently upgraded the chancing algorithm to reflect the most up-to-date information regarding admissions. You might see that some of your chances are now lower, especially for selective schools, and that is expected. If you notice something really funky going on, either email support@collegevine.com or reply to this thread to ping me.
Overview:
We've seen people wondering about the changes they are seeing in their chancing results. I'd like to explain the recent changes, why we made them, and what impact you can expect to see.
Upgrades to Chancing:
We have simultaneously updated the methodology of our algorithm and the data we use for the algorithm. In particular, this update is a paradigm shift in how we calculate your chances. Our updated model is a fully machine learning-forward approach to chancing, and we feel strongly that it is the best way forward. The transition to fully machine learning will allow us to more accurately capture the factors that affect your chances, and update our model quickly to reflect the most recent data that we have on admissions. It'll also let us more reliably predict chances for the very diverse set of profiles that CollegeVine users have.
Why We Make Upgrades:
At CollegeVine we strive to give you the most comprehensive college application guidance possible - for free. Part of that guidance is helping you, the college applicant, build a balanced school list by having realistic expectations for where you might get accepted. When better data and new methodologies are available, we reflect that new information in our algorithm to ensure that you're getting the best, most relevant, guidance possible. We realize that it can be jarring to see your chances change, and we are working on improving that experience for you. We apologize for any anxiety this has caused.
We also want to take this opportunity to remind everyone that although your chances might have decreased, all hope is not lost! We often see students feeling unhappy when they see low chances, and we think it is important to keep in mind that what you might view as a low chance is likely not a 0% chance. A 12% chance of admission, for example, still means you could get in! In fact, that's the same chance as you flipping a coin and getting tails three times in a row. Is it likely? No, but it also happens all the time, and you should try not to let your chances get you down. Instead, use your chances wisely to build a balanced school list and learn where you can improve your profile.
Expected Impacts:
As you may have noticed, your chances might be lower than they used to be. In some instances the chancing might be markedly lower. These changes will be most obvious for selective schools.
You may have also noticed that your profile might still be considered "excellent" or "strong" despite having significantly lower chances. This is because, especially in instances of selective schools, you still have a strong profile for the given school, but, because of the competitiveness these schools, your chance of acceptance is lower. For top-tier universities, chances can be in the "reach" range, even for students with excellent profiles.
How You Can Help:
Our team has thoroughly tested these upgrades, but if you believe you have results that are NOT expected, you can either email support@collegevine.com or ping me in this thread. We'll look into it!
Final Thoughts:
I'm expecting this thread to be popular based on the number of posts regarding this topic. Feel free to leave any questions, comments, thoughts, or concerns in this thread. I will respond to them here! I'd also love to hear feedback from the community on ways we can make the chancing rollout smoother in the future. What worked for you? What didn't work for you? How can we improve? Leave it all below!
This was really helpful. Thank you! :)
I really appreciate this update and its reflection of the difficulty in college admissions this past year! There are a few things in the chancing engine I have noticed that I wanted to point out to further improve accuracy (at least at select schools):
1. I changed my choice of major from Computer Science to Business Administration, Management and Operations. When I did so, my chances of admissions increased, which makes sense at schools since business tends to be less competitive than computer science. However, at Harvard for example, business is not offered as an undergrad degree. Because of this, it seems that the chancing engine should not have increased my chances since it would not be possible to apply to Harvard as a business major. In this case, I believe it may be helpful to have an option to add a secondary major to the chancing profile (Computer Science in my case). That way, the chancing engine would only update chances at colleges that actually offer the business degree. In addition, at some schools, I do not believe the chances would go up by applying as a business major. For example, Wharton school of business at UPenn is considered more competitive than applying outside of the school of business. In 2017, UPenn's acceptance rate was 9.2%, while Wharton's acceptance rate was 7.1%. At schools such as UPenn, applying to the business major should have actually lowered by admissions chances instead of raising them.
2. The mid-50 ACT/SAT ranges used by CollegeVine do not seem to take into account the mid-50 for the specific college (within a university) one is applying to. For example, CMU (Carnegie Mellon University) releases their ACT/SAT mid-50 by college (College of Computing, College of Engineering, etc.). After changing my numbers in the chancing profile, it seems that CollegeVine is using the same mid-50 across majors. However, for a student applying to CMU, they would only be compared to other students applying to the same school (School of Computing, School or Engineering, etc.), and so their mid-50 would be different than the average across CMU.
3. At some colleges, you do not declare a major when you apply. For example, at MIT, students are unable to declare a major until after they are already a student at MIT. However, the chancing engine is still updating my chances when I change my major on my profile.
4. The chancing engine does not seem to take into account super-scoring policies on ACT/SAT. As a future upgrade, I think it would be great if it was possible to enter scores from multiple ACT/SAT sittings. That way, CollegeVine could use my super-score to calculate chances at colleges that accept super-score and my highest composite at colleges that don't.
5. As another future improvement, I think it would be helpful to have an option to indicate if we are applying EA, ED, or RD. I know that applying ED at many schools results in a higher chance of admission (since you already committed to going if accepted), so I think this distinction would have different results in the chancing engine.
Once again, thank you for your work to make the chancing engine such a reliable tool!
ED would just be fairly difficult to model. Its like giving someone 25 random stocks and expect them to model the entire stock market. Like 500 isn't even a great indicator but its not horrid but take that down to 25 ouch.
Also @ShaquilleOatmeal could you have a way to indicate if schools super score so that way you can plug in super score data?
@DebaterMAX sorry for the delay in getting back to you. This is a good suggestion, I'll pass it along to the team!
@CollegeAdmissions you make a lot of good points here! All of the things you mentioned are on our radar and are things we are considering for potential improvements down the line.
"Our updated model is a fully machine learning-forward approach to chancing, and we feel strongly that it is the best way forward"
Machine learning is data-driven. Where are you getting the data from, how accurate is it? Machine learning algorithms really only provide accurate data when the data is stable and you have a lot of data points but college admissions are kind of the opposite. How do you account for this?
Hi @lij1207! I’m Matt, and I’m a data scientist here at CV and one of the major players behind our revamped chancing algorithm.
First, thanks for this question — talking about data quality, algorithm reliability, and bias in machine learning is extremely important.
With that said, I want to start by saying how proud I am of our new approach and model. I think we’ve put together a model that does a really great job of combining machine learning with our expert knowledge of college admissions to predict your chances — it’s our best one yet.
You’re completely right to bring up concerns about ML being data-driven, and you’re right that ML algorithms can learn about all kinds of biases in the data they’re trained on and end up making garbage predictions. Fortunately, in our case, we’re overcoming these known issues with a number of checks that should be part of any ML modeling process: we check our data itself, and, most importantly, we check the performance of the model on data that it hasn’t seen yet to make sure its predictions are well calibrated and are not biased. These types of checks are a critical piece of our modeling process at CV, as we want to do everything in our power to ensure that the chances we’re showing students who are relying on them are as accurate as they can be.
To your data question specifically: we have lots of diverse data on admissions results in recent years that we use to train the model, meaning that we’re able to model admissions chances for students from all backgrounds applying to all different types of schools with confidence. The amount and diversity of our data makes me even more confident in how our model is representing chancing in the real world.
The last thing we’ve done here is that we’ve built the vast majority of our expert knowledge — which used to make up our old chancing algorithm — into this model. This means that our new ML model is actually relying heavily on the same expert knowledge that we’ve built up over time here at CV, and the model is made to reflect this expert knowledge of how admissions works.
Thanks again for your questions and concerns! Hopefully this explanation helps you be more confident in how we’re modeling your chances. I know that ML can be intimidating and is often misused, and it’s important to us at CV that we do everything in our power to use transparent, fair machine learning for good.
Where all are you getting the data from? Buying it? Common data set?
> To your data question specifically: we have lots of diverse data on admissions results in recent years that we use to train the model.
@DebaterMAX Our data comes from a variety of sources, but we’re unfortunately not able to discuss individual ones. Sorry :/
Ah. Can you share any that are public ally available?
@matt.kaye
@DebaterMAX Unfortunately can't say on that either. Sorry again.
Separately though -- as a person who works with data every day, I think it's awesome that you're interested in using open-source data for research purposes!
Thank you for making these changes! However, I would love to see some changes to the way IB and honors are calculated. Indivudal boxes for honors courses offered, and a check box for full ib students. It doesnt really work to fill out how many classes are offered at our school for ib because IB has so many limitations to how many courses you can take.
I agree, the Coursework area needs to better address IB/AP classes and perhaps even make a distinction between the two. The IB diploma program does not allow you to play to your strengths in contrast to how simply taking AP or isolated IB classes would permit (not to mention the insane amount of extra work necessary to obtain the diploma). Thus, the Coursework section could be improved by creating further distinctions between the number of IB/AP courses taken, versus earning a certificate, versus obtaining the full IB diploma and/or AP Capstone.
For example, by the time I am a senior, I will have taken 13 IB/AP classes, 5 honors ("pre-IB classes"), and completed the IB diploma; however, I'm rated only as "strong" for selective schools for Coursework. This doesn't seem to make sense to me given the rigor and given the limitations of how the IB program works in general. (With an exception to double jump math students at our school, you can't take IB classes your freshman or sophomore years). In other words, short of adding summer school classes, there is very little else I could do to make my courseload more rigorous within the confines of the IB program.
Additionally, IB diploma credit aside, there are tiered distinctions between SL and HL IB classes. Would admissions officers consider HL math taken as a sophomore more rigorous than an SL math as a junior or senior? Perhaps if the GPA section could weight our GPA at least in core classes, this might better reflect what our academic index might really be?
In any case, I was feeling much more confident about several of the schools I was looking at before the chancing algorithm adjustment. Now, many of the schools have shifted from "targets" to "reaches" which is really disheartening. In any case, I guess it would be really good to know if pursuing the IB diploma is recognized as highly rigorous by admissions officers. If so, then the chancing estimator needs to better reflect that.
Hello!!
I really like the new updates! However, for my target school, it used to show a 30-40% chance of getting in and now it shows a 60-70% chance. Is this normal/accurate or can this possibly be an overestimate?
Thank you! :)
What is/was your one word score fit each section (excellent strong needs more work) also what school? If you don’t mind me asking
Hi @shridha! This is totally normal! You're probably seeing lots of posts about chances going down, but the reality is that we were actually expecting different people to have their chances change in different ways. Nothing to worry about here :)
Hey! It was excellent for all categories before and after the updates, and the school is UT at Austin
Ahh I see, thank you for clarifying! :)
Your chance of acceptance
Your chancing factors
Community Guidelines
To keep this community safe and supportive:
- Be kind and respectful!
- Keep posts relevant to college admissions and high school.
- Don’t ask “chance-me” questions. Use CollegeVine’s chancing instead!