0
$\begingroup$

Using the methodology described in FiveThirtyEight.com's "How We Found the Fastest Flights" in the footnote:

A simplified formula for target time is as follows:

.117*distance + .517*(longitude of origin – longitude of destination) + 43.2

The formula produces an estimated travel time in minutes. In the formula, distance is the great circle distance converted to miles between the origin and destination airports. The coefficient next to distance (.117) implies that flights travel at 513 miles per hour at cruising speeds.

Longitude origin and longitude destination represent the longitude of the airport of origin and destination, respectively. Each degree of longitude traversed westbound adds about 30 seconds to the flight time.

The constant term, 43.2 minutes, reflects the time airlines budget into their schedules for taxiing in and out of the airport, plus potentially inefficient routing.

The more precise version of target time reflects that degrees of longitude are further apart near the equator, and that the jet stream is slightly stronger at middle latitudes. It is calculated as follows:

.117*distance – .0255*westmiles+ .00144*westmiles*(90-latitude of midpoint)^2 – .0000146*westmiles*(90-latitude of midpoint)^2 + 43.2

In this formula, latitude of midpoint is the latitude at the midpoint of the journey (using a great circle route) and westmiles is the number of miles traveled westbound. Westmiles is calculated with this formula:

(longitude of origin – longitude of destination)*-.0073*(90 – latitude of midpoint)^2+1.424*(90 – latitude of midpoint)

Here's a link to my Google sheet where you can see the formulas in action, but I have also tried to explain it below. The example in the methodology is San Francisco airport (SFO) to Los Angeles airport (LAX). The inputs were not provided so I have come up with the following using Google maps and the Great-Circle Distance calculator:

SFO latitude           37.6213171

SFO longitude        -122.3811494

LAX latitude           33.9459649

LAX longitude        -118.4061815

Latitude midpoint      35

distance (miles)      338

Longitude origin      122

Longitude destination 118

To calculate the variable westmiles:

(longitude of origin – longitude of destination)*-.0073*(90-latitude of midpoint)^2+1.424*(90 – latitude of midpoint)

Which I interpreted as:

(122 - 188)*-.0073*(90 - 35)^2 _ 1.424* (90 - 35)

and it equals -10 (I don't know if that is correct).

Putting the detailed formula together, I'm supposed to get 1 hour 19 min, but instead I got 40 min. 

.117*distance - .0255*westmiles+ .00144*westmiles*(90-latitude of midpoint)^2 - .0000146*westmiles*(90-latitude of midpoint)^2 + 43.2

This is how I interpreted it:

.117 * 338 - .0255 * (-10) + .00144 * (-10) * (90 - 35)^2 - .0000146 * (-10) * (90 - 35)^2 + 43.2

I believe ^2 means to the power of two, but I could be wrong. Can I get feedback where I went wrong? Or does someone have an alternative to this formula?

$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Make sure you are using negative longitudes in all your formulas, because your cities are in the Western Hemisphere. So in your example, Westmiles equals -223.5 which makes sense in this context.

Another useful formula for the calulation of east-west lengths at a specific latitude is the following: $$\left(\lambda_o - \lambda_d\right) r \text{ cos }\phi$$ Where $\lambda_o$ and $\lambda_d$ are your longitudes of origin and destination (in radians), $r$ is the Earth's radius and $\phi$ is the latitude.

$\endgroup$
7
  • $\begingroup$ If I use negative longitude origin and a negative latitude origin, my westmiles variable is 167. Google sheet docs.google.com/spreadsheets/d/… $\endgroup$
    – adelaide01
    Mar 15, 2018 at 23:31
  • $\begingroup$ There should be a parenthesis before -0.0073 and one at the end, like this : (lon_origin-lon_dest)*(-.0073*(90-lat_midpoint)^2+1.424*(90-lat_midpoint)). They seem to have made a typo on fivethirtyeight.com because the formula does not make sense without those parentheses. $\endgroup$
    – FSimardGIS
    Mar 16, 2018 at 20:42
  • $\begingroup$ Moreover, with the simpler formula that I posted in my answer, the result is -225.04. 167 just wouldn't make sense in your example, because the plane is going eastward. $\endgroup$
    – FSimardGIS
    Mar 16, 2018 at 20:50
  • $\begingroup$ OK. I have -225 for westmiles now following your advice about where to put the parenthesis, but plugging in westmiles into the equation results in -881. It's supposed to be 94, as in 1 hour and 34 min. What else do I need to do to get there? $\endgroup$
    – adelaide01
    Mar 17, 2018 at 22:23
  • $\begingroup$ Apparently, they made a mistake in their precise version equation too. It should be .117*distance – .0255*westmiles+ .00144*westmiles*(90-lat_midpoint) – .0000146*westmiles*(90-lat_midpoint)^2 + 43.2. There shouldn't be a ^2 after the 3rd term. This way it fits with the description that they provided. Using this I arrive at 1 h 20 min. $\endgroup$
    – FSimardGIS
    Mar 18, 2018 at 4:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .