Essentially, I have a slightly weak application in terms of AP/Honors coursework so I am leaning against applying to the COE, but I want to know whether that should deter me.
What do you want to do with what you study? Berkeley is difficult to get into no matter what—the difference in difficulty between COE and CLS really will not be as significant as it might feel, and their academic thresholds in particular will be largely the same (i.e., if you're just worried about a lack of AP courses, applying to CLS vs. COE shouldn't make much of a difference). You should apply to the one that a) better fits your goals for the future and b) better fits what your profile. If you want to be an engineer and have the career opportunities associated with that—basically, if you want to build things, design things, work for engineering firms, etc., you should apply to COE and aim to study Engineering Physics. If you're more interested in pure scientific research (either in industry with a B.S. or after further graduate school) or physics as a platform to more typical jobs, go for General Physics. Even though they seem similar, there is a real difference between what each curriculum will set up for you, and that should imo be your biggest consideration when you aim for one.
(Incidentally, Berkeley is set up in a way that makes transferring between "impacted majors," of which all of their engineering majors typically are. So if you want to become an engineer, applying into CLS and attempting to internal transfer may not ultimately work out, no matter how good you are, if all of their spots are taken.)
What do your ECs look like? How easy is it to transfer in and out of each major? I think you should consider that before choosing.
To keep this community safe and supportive:
Can't I become an engineer with a pure physics degree? and then maybe grad school for EE/ME?