All-Star Comparing Scores Across Competitions

[imrichhowboutu]

kingston - perhaps the issue isn't EPs or Safety Judges, perhaps it is coaches? If coaches knew the rules front to back and didn't have illegal skills in routines, Safety judges would only be looking for performance errors. I think that all coaches should be REQUIRED to pass safety certification in order to get coaching certification for a specific level. It would be better for everyone all around, ESPECIALLY the athletes.

My biggest issue is when coaches blame Safety Judges for a loss and how it affects the athletes, but they don't seem to care that they have been ignoring safety rules for the specific level and could potentially put their athletes in danger.

 

[Hungry]

I don't panel judge too often anymore. I can say from personal experience in coaching and from friends that I have seen tougher execution scores at larger nationals. The thing you have to realize is that when you go to a competition with 1 team in a division, vs a competition with 20 in a division, a lot has changed and that 0.1 is going to make all of the difference.

What you say makes sense but both these competitions were small (3 in the division first comp, two in the second) so as a coach, I'm trying to wrap my head around why tech scores were so much different so that I can understand and improve.

I wish I could see the tech scores for my competitor to see if tech was scored low across the board.

Thank you for your feedback!

 

[imrichhowboutu]

What you say makes sense but both these competitions were small (3 in the division first comp, two in the second) so as a coach, I'm trying to wrap my head around why tech scores were so much different so that I can understand and improve.

I wish I could see the tech scores for my competitor to see if tech was scored low across the board.

Thank you for your feedback!

There is no way to say what happened. Different judges have different criteria. Head judges and other processes are in place to ensure things are consistent.

 

[NJ Coach]

And this is why we need a coaches room! (see the pet peeves thread) At Athletic in Providence, I was able to talk to lots of coaches, and all were talking about the low scores received (both category and technical). I wasn't happy when I first got my scores, but by day 2 when I realized no one else was either, I cheered up

 

[yojaehs]

I feel that judging, safety judging, choreography and music should all have separate associations responsible for their part of routines.
If there is a judging mistake the blame should lie within their association, same with safety judges, same with choreography (if an inappropriate dance, illegal pyramid etc) same with music (if language, voice over etc.).
Now I will agree that choreography/music should lie more on the responsibility of the coaches but at the same time there are TONS of gyms getting taken advantage of. Do I wish more coaches were knowledgeable and did better research? YES. But do I feel that choreographers are getting out of control, YES. Do I feel that music is getting out of control, YES. There is NOTHING out there to protect anyone from being screwed by a choreographer overcharging, not showing up, or providing illegal material. Same goes for music, I hear DAILY of coaches being screwed by music producers. Now, again, the responsibility should lie with the coach, but I do wish there was something to protect coaches against shady choreographers/music producers.

Ok back to judges/safety judges. I think of it like this, if I go to a football, baseball, hockey game or gymnastics meet, and a bad call or bad score comes up, who do I blame? Example: Superbowl - Ravens vs Niners there is a holding call within the last minute of the game (not called based on referee choosing not to call) this is debated nationwide. Does the blame lie on the event of the Superbowl? NO, the responsibility lies on the referee and will be reviewed and discussed to improve referees going forward. I feel like EP XYZ could put on the BEST event possible and everything goes great, then the scores come out and BOOM everyone HATES that EP. A judging association would solve a LOT of problems.

I know I'm asking for a lot and probably won't see any of this ever happen BUT I'd rather see USASF put their focus on judges and less on a dang cover up rule that again, is the STUPIDEST thing I've ever heard of.

Ok, off of soapbox...for now.

 

[newcheerdad]

human error doesnt bother me as much as motivation. i think outside of pride in ones job there is no reason for safety judges to get it right OR for EP's to enforce the calls. do i think fining every penalty is realistic? no. do i think we need to find some way to motivate and have consequences if a penalty is missed? yes. all the teams currently have a penalty if missed.

This is why the current model is screwy. You need a couple of big things to make this work better:

1. Judges shouldn't be certified directly by an event producer. There should be a organization that trains and certifies judges independent from the EP's. EP's would be required to use judges certified by this body. (Funding would come from EP's along with member dues, but EP's would have minimal influence) That organization would evaluate the performance of judges and grade them accordingly, not to dissimilar to how soccer works. As a USSF referee, if I want to work higher level competitions, I need to be a higher graded official. To get there, I need to show demonstrated experience at lower level competitions plus be evaluated by USSF-certified assessors. A system like that can work in cheer if you think outside the box a little bit.

2. You need common scoresheets and rubrics at these competitions. If you want judges to really understand what they're scoring and why, they should be using the same scoresheet every time. You can't tell me that the multitude of different rubrics out there doesn't contribute to these discrepancies in scoring.

3. I agree the mechanics of judging could be changed. I think there should be separate judges for difficulty and legality as opposed to technique/execution and presentation. It's separation of duties, again not too dissimilar to the roles that a center and assistant referee have in soccer. That doesn't mean that the technique judge can't bring up a legality question (just like an AR can signal for a foul in soccer), but the responsibility for legality or difficulty is the responsiblity of a particular person. Again - if there was a common standard for judging, common certification and common scoresheets, you would have judges trained this system and better able to do their jobs.

Sorry if that's a ramble, but I think a lot of this tacks back to the obvious issue, which is the lack of a really strong governing body for cheer.

 

[cheerinfo]

newcheerdad: Safety judges are separate from difficulty/panel judges. They are also not certified by EPs, they are certified through the USASF, and are re-certified each year (if they pass the test). What I think part of the problem is that EPs all use different scoresheets (not ALL, but i mean, there is not a universal scoresheet) with different parameters, different amounts to be used towards the different categories, etc. And also, in the end, panel judges have a certain amount of subjectivity in scoring, whereas safety doesn't so much. The fact with safety judges is that you just can't always get everything, every weekend, every day, all the time. No amount of penalizing or testing or throwing tantrums will change that. And panel judges may make mistakes too. Judges are human. Human error exists, and there is no cure all for fixing that. We can just be as diligent as possible and go from there. Just wanted to let people know that USASF does certify safety judges. I don't know of any major events that would or do use judges that are not on that national list of safety judges that pass the exam each year.

 

[yojaehs]

newcheerdad: Safety judges are separate from difficulty/panel judges. They are also not certified by EPs, they are certified through the USASF, and are re-certified each year (if they pass the test). What I think part of the problem is that EPs all use different scoresheets (not ALL, but i mean, there is not a universal scoresheet) with different parameters, different amounts to be used towards the different categories, etc. And also, in the end, panel judges have a certain amount of subjectivity in scoring, whereas safety doesn't so much. The fact with safety judges is that you just can't always get everything, every weekend, every day, all the time. No amount of penalizing or testing or throwing tantrums will change that. And panel judges may make mistakes too. Judges are human. Human error exists, and there is no cure all for fixing that. We can just be as diligent as possible and go from there. Just wanted to let people know that USASF does certify safety judges. I don't know of any major events that would or do use judges that are not on that national list of safety judges that pass the exam each year.

I think Jamfest "score check" system is fantastic. It is an immediate "range" and "safety" sheet showing the coach which ranges they were placed in as well as if a safety violation was caught. In addition a video review is provided if coaches would like clarification or to challenge. If I was a "safety judge" I would request that I be put separate from "judges". If your job is to catch safety violations then don't be surprised if I get pissed that you missed YOUR JOB. You get paid from the EP's that UPCHARGE me for your services. I don't care the size of the comp but if you miss it then I get hit later, A. It's #1 my fault for having it in there B. I'm gonna be PISSED at the safety judge that missed it bc I paid to have it caught early rather than late.

 

[newcheerdad]

newcheerdad: Safety judges are separate from difficulty/panel judges. They are also not certified by EPs, they are certified through the USASF, and are re-certified each year (if they pass the test). What I think part of the problem is that EPs all use different scoresheets (not ALL, but i mean, there is not a universal scoresheet) with different parameters, different amounts to be used towards the different categories, etc. And also, in the end, panel judges have a certain amount of subjectivity in scoring, whereas safety doesn't so much. The fact with safety judges is that you just can't always get everything, every weekend, every day, all the time. No amount of penalizing or testing or throwing tantrums will change that. And panel judges may make mistakes too. Judges are human. Human error exists, and there is no cure all for fixing that. We can just be as diligent as possible and go from there. Just wanted to let people know that USASF does certify safety judges. I don't know of any major events that would or do use judges that are not on that national list of safety judges that pass the exam each year.

I know that USASF certifies safety judges. But I think the USASF, or some body sanctioned by the USASF, should be certifying ALL judges. And those judges should be working from a common rubric and set of parameters by which they can be evaluated. As a soccer referee, when I'm evaluated for an upgrade, there's a set of parameters that I can be evaluated on no matter what type of game I'm refereeing because the basic rules and mechanics are the same across the board. That really isn't the case in cheer judging, because those can apparently vary from EP to EP.

I get that there's a human element to some of this, and that even if you implement a common scoresheet, standardized training and frequent evaluations you'd still have issues from time-to-time. But what it would do is move the onus for those issues from the EP to an external body whose job it would be to make their judging better and more consistent.

 

[cheerinfo]

I think Jamfest "score check" system is fantastic. It is an immediate "range" and "safety" sheet showing the coach which ranges they were placed in as well as if a safety violation was caught. In addition a video review is provided if coaches would like clarification or to challenge. If I was a "safety judge" I would request that I be put separate from "judges". If your job is to catch safety violations then don't be surprised if I get pissed that you missed YOUR JOB. You get paid from the EP's that UPCHARGE me for your services. I don't care the size of the comp but if you miss it then I get hit later, A. It's #1 my fault for having it in there B. I'm gonna be PISSED at the safety judge that missed it bc I paid to have it caught early rather than late.

I don't disagree. There are many companies that have video verification, etc. And I agree that it is frustrating when something is missed. I am just saying that nothing will get rid of that human error entirely. That was my only point. Fining people, yelling at people, whatever, they are still people and will still make mistakes from time to time. I think there are ways to better the system, but that aspect will always be there in some way, shape or form.

 

[newcheerdad]

I think Jamfest "score check" system is fantastic. It is an immediate "range" and "safety" sheet showing the coach which ranges they were placed in as well as if a safety violation was caught. In addition a video review is provided if coaches would like clarification or to challenge. If I was a "safety judge" I would request that I be put separate from "judges". If your job is to catch safety violations then don't be surprised if I get pissed that you missed YOUR JOB. You get paid from the EP's that UPCHARGE me for your services. I don't care the size of the comp but if you miss it then I get hit later, A. It's #1 my fault for having it in there B. I'm gonna be PISSED at the safety judge that missed it bc I paid to have it caught early rather than late.

Do I think you need to have video review and score checks at every competition? I can argue that point either way. But I think my bigger point is that the points of the scoresheet that are more objective, like assessing difficulty, should be in the hands of someone who's responsible for getting it right and will be evaluated on their ability to get it right.

 

[BlueCat]

I have always thought that the scoring could/should be divided between the subjective and (theoretically) objective portions of the scoresheet. There are ways to make the objective part more accurate, and I think every reasonable effort should be made to make that happen. (video review, separate difficulty judges, skills declarations turned in ahead of time, detailed rubrics, etc.)

NOTE: This is NOT suggesting that we need to have cookie-cutter routines or that everything should be objective. I just want the objective parts to be scored correctly. There will still be mistakes made, but they should be few and far between.

As I may have mentioned previously, the biggest push to improve scoring would be to open the hidden elements up to the public. Let everyone see all of the scores. That is the only way, IMO, to be able to fairly judge the judges.

 

[newcheerdad]

I have always thought that the scoring could/should be divided between the subjective and (theoretically) objective portions of the scoresheet. There are ways to make the objective part more accurate, and I think every reasonable effort should be made to make that happen. (video review, separate difficulty judges, skills declarations turned in ahead of time, detailed rubrics, etc.)

NOTE: This is NOT suggesting that we need to have cookie-cutter routines or that everything should be objective. I just want the objective parts to be scored correctly. There will still be mistakes made, but they should be few and far between.

As I may have mentioned previously, the biggest push to improve scoring would be to open the hidden elements up to the public. Let everyone see all of the scores. That is the only way, IMO, to be able to fairly judge the judges.

There should be a human element in scoring when it comes to judging the presentation of the routine and the technique of the skills executed. But the difficulty of the routine and the assessment of safety violations should be as accurate as possible.

I've always thought that full breakdowns of scores - or even breakdowns of specific score elements (i.e., tumbling, stunts) should be available. But even if you don't do that, at a minimum there should be people outside of the event producer that review scoring and assess whether said scoring is reasonably accurate.

 

[5-6-7-8]

.

As I may have mentioned previously, the biggest push to improve scoring would be to open the hidden elements up to the public. Let everyone see all of the scores. That is the only way, IMO, to be able to fairly judge the judges.

I agree with all of what you wrote but this specifically. Opening up the scores for all to see will force accountability really quickly.

I don't know about everyone else but if I were judging knowing scores would be made visible, I would definitely putting even more effort into getting it right.

And I am not saying judges don't want to make it right I believe they do. But with any job, if I knew I was going to be audited for my work I'd want to get it right.

 

[yojaehs]

I agree with everything BlueCat said and I've always made MY teams score sheets accessible to anyone interested. I believe that if there were a Judges Association the EP would be able to focus on the event itself and not have to worry about the judging side.