I can help with part of it. If you let theta1 and theta2 be both from the positive x-axis, then the three points are at:
P0 = (1,0)
P1 = (cos(theta1), sin(theta1))
P2 = (cos(theta2), sin(theta2))
You can then easily get the vectors P0P1 and P0P2, and the area of the triangle is half the determinant of those two vectors.
As for max area, if you use the symmetry argument that max area happens when theta2 = -theta1, you can reduce the kind of complicated area formula to something simpler with only one variable:
A = sin(theta) - sin(theta)cos(theta)
- or -
A = sin(theta) - (1/2)sin(2*theta)
(You can probably see this more easily drawing the picture and going with straight 1/2*base*hieght, forget the vector stuff.)
If you differentiate w/r to theta and set to zero, you get:
cos(theta) = cos(2*theta)
which is satisfied by theta = 2*pi/3 and it is indeed an equilateral triangle.