Manfred Talks Robot Umps - Tech is "Way Up"

Echoing 2017 comments downplaying opposition to an electronic strike zone as a replacement for the home plate umpire's pitch-calling role, MLB Commissioner Rob Manfred painted a picture welcoming robot umpires, explaining that improving technology has led to greater accuracy and consistency in the computers' own pitch-calling abilities.

Manfred in 2017 initially stated that he opposed implementation of an electronic strike zone during live play, but appeared to conduct an about-face just days later, explaining that when computer accuracy surpasses human umpire accuracy regarding balls and strikes, it might be time to think of a change.
Related Post: Virtual Reality for Umpires & Manfred's Strike Zone (8/24/17).

Video: Ump Show Video Game (4/1/18). Coming soon?

Explained Manfred this season, "The accuracy is way up—way better than what it was a year ago. The technology continues to move...and it actually moved a little faster than I might have thought."

The precise numbers—other than the human umpires' greater-than-95% accuracy figure—were not disclosed, though, in an apparent, though potentially inadvertent, snub and potshot at SMT Sportvision, which this year sued MLBAM alleging it stole PITCHf/x technology in the use of Statcast and Trackman pitch tracking services, Manfred said that, "we have worked very hard on PITCHf/x": not MLBAM's "PITCHcast," not Statcast...but SMT Sportvision's product name, "PITCHf/x."

According to SMT/Sportvision's lawsuit, MLBAM breached its contract by continuing to reap the benefits of PITCHf/x under the rebranded title PITCHcast, without compensating SMT for its proprietary technology and trade secrets, also constituting theft and patent infringement.
Related Post: Pitch f/x SMT Sportvision Sues MLBAM for StatCast 'Theft' (5/21/18).

Rob Manfred praised pitch-tracking technology.

Manfred's comments seems to have gotten some players thinking: after hearing Manfred's praise of electronic strike zone technology, Boston media reported that pitchers David Price and Drew Pomeranz "are curious about [the] possibility." Said Price, "It would be a consistent strike so that's good. But it would be kind of weird at first. I like having the umpire. I understand how tough the job they have to do is, especially the guys behind home plate."

Pomeranz didn't want to take human umpires entirely out of the equation either, and proposed a compromise: "I could see them going to some kind of hybrid."

Added reliever Heath Hembree, "At the end of the day everything evens out with an umpire behind the plate now and it'll still even out with a robot."

---

Gil's Call: Not withstanding arguments concerning the art of pitch framing or how a myriad of "expected" calls will no longer be called as "expected" if technology makes its way behind the plate, the premise of judging accuracy relative to an electronic strike zone is somewhat flawed, with a heavy dose of circular logic. The problem with judging human and computer strike zone accuracy is, largely, one and the same.

Pitch f/x, Zone Evaluation, Statcast, Trackman, and PITCHcast are all cut from the same elemental cloth: at each product's basic level, there is an underlying reliance on electronic measurement and adjudication based on algorithms and other computing properties.

Accordingly, we're asking a computer to grade both humans and to grade itself. While having a computer grade humans is an acceptable, albeit flawed, practice, asking a computer to grade itself exposes and compounds the holes and errors in the technology itself—how is a computer to know when it has missed a pitch if it thinks it got the call right in the first place? By the same token, how is a human umpire to know when he has missed a pitch if the umpire thinks the pitch was correctly called in the first place?

The answer is a blend of human-and-tech, which MLB first attempted to solve with QuesTec, then Zone Evaluation, and now, whatever they want to call that which is being contested in SMT's lawsuit.

When MLB adopted Statcast, the vertical errors only increased.
Graph: FiveThirtyEight.

For instance, the computer's accuracy Manfred refers to may pertain to the horizontal bounds of the strike zone—whether the pitch nicked the inner/outer edge of home plate or not—but the computer still doesn't have a great idea of how to measure the vertical properties of the zone: while the horizontal variable of home plate remains constant at 17" across + two 2.94" diameters of a baseball (= 22.88" working strike zone), the vertical varies at-bat to at-bat, and even pitch to pitch (right now, the computer uses "averages" and generally fails to adjust these average sz_bot and sz_top values once a game has begun, no matter how the appropriate batting stance evolves over the course of nine innings).

Still, the data refutes Manfred's claim of "way up" accuracy—see accompanying graph for details.

Based on what happened to vertical error when MLB changed PITCHf/x to PITCHcast in 2017 (error increased significantly), Manfred's claim that the technology is "way up" in accuracy may, itself, be misleading: if Manfred is comparing "a year ago" to the 2018 season, since we have no publically-distributed figures for 2018, Manfred could theoretically refer to a decrease in vertical error caused in the first place by MLB's own decision to switch from PITCHf/x to PITCHcast—a net gain/loss of zero when compared to 2016 and PITCHf/x.

The claims are far too vague to mean all that much.

Then there's the 3D zone and other issues of error we've discussed ad nauseam, but the overarching theme and conclusion here is that the electronic vertical strike zone still needs a lot of work.
Related Post: Dude, What Happened Last Night? About Pitch f/x Error (8/30/16).
Related Post: Analyzing Strike Zone Analysis - Not So Easy or Simple (10/27/16).

One of these days, they'll surely figure it out, but we're certainly not there yet.