I am trying to calculate the angle between two latitudes and longitudes but I am getting wrong result of atan2(X,Y) when used on Excel.

β = atan2(X,Y) = atan2(0.05967668696, -0.00681261948)

It should be 1.68radian (96.51°) but on excel it is showing -0.113666736 (-6.512624235°)

β = atan2(X,Y),

where, X and Y are two quantities and can be calculated as:

  • X = cos θb * sin ∆L
  • Y = cos θa * sin θb – sin θa * cos θb * cos ∆L

    Latitude/Longitude(1): (Lat1,Lon1)= 39.099912, -94.581213

    Latitude/Longitude(2): (Lat2,Lon2)= 38.627089, -90.200203

    ‘L’            be the longitude,
    ‘∆L=lon2-lon1’ be the difference of longitude,
    ‘θ’            be latitude,
    ‘β‘            be Bearing
  • Not sure if you're aware, but there is a mathematica.se - also my atan2() is giving me -0.11367 - so you're converting it to degrees, which you didn't indicate. – Raystafarian Aug 3 '15 at 18:04
  • yes you are right, I am converting it in degree after using atan2(x,y).. – Fahadkalis Aug 3 '15 at 18:10
  • that's the same formula I am using – Fahadkalis Aug 3 '15 at 18:24
  1. There is nothing wrong with atan2(X,Y) function in Excel: the point you have specified '(0.05967668696, -0.00681261948)' lies in the fourth quadrant, i.e. its angle is really '-6.51°'.

  2. But if you want calculate an 'absolute' value of angle between two geographical points, you should use and absolute difference between coordinates:

      β=ATAN2(ABS(0.05967668696), ABS(-0.00681261948))
  3. Bear in mind that if this calculation is on a sphere (Earth), it is an approximation because of circular triangle. It is OK for small angles.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.