Still studying navigation and am trying to consolidate my knowledge of the navigational triangle. Current interest in particular is how to establish the distance of the unknown side of the PZX triangle. From the assumed position of the ship (Z) to the geographical location of a star (X).

Lets throw in a couple of positions to work with:

ASSUMED POSITION Z
54 degrees 13'.45N
003 degrees 14'.62E

POSITION OF STAR X
47 degrees 17'.95N
019 degrees 37'.82W

I am familiar with the following formula as a result of help received on Jiskha, thanks Damon.

cosp = cosx x cosz + sinx x sinz x cosP

and a minute or so on the calculator will yield an answer.

I have now become curious about another formula I have come across to calculate the same information and need a lot of help AGAIN! applying this new formula.

New formula as follows:

Distance=60.ACOS(sin(latX)x sin(latZ)+cos(latX)x cos(latZ)x cos(longX - longZ))

Too many brackets for my liking. Obviously if both formulas are for the same purpose they should yield the same answer working with positions given above.

Please can someone have a go at the calculations using both formulas to see if it is possible to achieve the same answers.

I would also be interested to learn of any other formulas suitable for calculating distances between 2 positions on earth.

Thanks a million

Mike

The first one is the formula I used yesterday and agreed with your answer. I also checked it against the online navigation calculators.

However I remember you did something like take 90 - the latitude before you ran your formula.

If you did that, then since cos (90-x) = sin (x) and sin(90-x) = cos (x)
the two formulas would be the same.

The 60 is to get nautical miles instead of degrees.

For the data you gave above the online software gives 956.7 nm

I will calculate roughly using your data and the second formula since I did not use that yesterday. I am dividing minutes by 60 to get fractional degrees for my calculator which does not do base 60 arithmetic.:
sin Lat X = sin 47.299 = .7349
sin Lat Z = sin 54.224 = .8113

cos Lat X = .6782
cos Lat Z = .5846

cos (long X -long Z) = cos (19.630+3.244) = cos(22.874) = .9214
note: The Plus sign here is because the Z longitude is East - - =+
so
cos^-1[ .7349 * .8113 + .6782 * .5846 * .9214 ]

cos^-1 [ .5962 + .3653 ]

cos^-1 [ .9615 ]

15.95 degrees

times 60 = 957 nm

So the second equation agrees with the online software navigator

Go to wikipedia on the subject of great circle navigation. They have several of the equations, the one you just gave being on the top. They give examples of where one might be accurate and another inaccurate.

Here is an excerpt from the online aviation navigation primer. If it prints here, it also shows your second formula as first choice and the haversine formula second:

Some great circle formulae:
Distance between points

The great circle distance d between two points with coordinates {lat1,lon1} and {lat2,lon2} is given by:

d=acos(sin(lat1)*sin(lat2)+cos(lat1)*cos(lat2)*cos(lon1-lon2))

A mathematically equivalent formula, which is less subject to rounding error for short distances is:

d=2*asin(sqrt((sin((lat1-lat2)/2))^2 +
cos(lat1)*cos(lat2)*(sin((lon1-lon2)/2))^2))

No one is listing the one I had from my old books and I can not get an accurate result from it on your second problem. I suggest using the ones I am seeing in the present day texts.

The Wikipedia article on marine navigation and the aviation source agree on the
cos^-1 [sin sin + cos cos cos] form, so I would stick to that.
They also agree on the haversine form, but I do not think you really need that unless just for fun. You can always carry more places with your calculator if you want more accuracy for special cases.
By the way, the online calculators allow you to use different earth models. We are assuming a perfect sphere as have navigators for centuries. However greater accuracy is possible using more accurate models for the geometry of earth, but no way you are going to figure them out with a pocket calculator.

To establish the distance of the unknown side of the PZX triangle, you can use the haversine formula or the Spherical Law of Cosines formula. Let's calculate the distance using both formulas for the given positions.

First, let's convert the degrees, minutes, and seconds into decimal degrees:

ASSUMED POSITION Z:
Latitude: 54.22417°N
Longitude: -3.24367°E (converted to -3.24367°)

POSITION OF STAR X:
Latitude: 47.29917°N
Longitude: -19.63033°W (converted to -19.63033°)

Now let's calculate the distance using both formulas:

1. Haversine formula:
The haversine formula is given by:
Distance = 2r arcsin(sqrt(sin^2((latX-latZ)/2) + cos(latX)cos(latZ)sin^2((longX-longZ)/2))))

where r is the Earth's radius (mean radius = 6,371 km or 3,959 miles).

Using this formula:

Distance = 2 * 3959 * arcsin(sqrt(sin^2((47.29917-54.22417)/2) + cos(47.29917)cos(54.22417)sin^2((-19.63033-(-3.24367))/2))))

Calculating this on a calculator will give you the distance between the two positions using the haversine formula.

2. Spherical Law of Cosines formula:
The Spherical Law of Cosines formula is given by:
Distance = acos(sin(latX)sin(latZ) + cos(latX)cos(latZ)cos(longX-longZ)) * r

Using this formula:

Distance = acos(sin(47.29917)sin(54.22417) + cos(47.29917)cos(54.22417)cos((-19.63033)-(-3.24367)))) * 3959

Calculating this on a calculator will give you the distance between the two positions using the Spherical Law of Cosines formula.

Both formulas should yield approximately the same result, which is the distance between the assumed position Z and the position of star X.