•  Why was oxygen used as a reference element to measure atomic mass initially? Then why was it rejected?
• I read somewhere that hydrogen was used as a reference element even before oxygen came to be used. Is it true?

1) The reference was basically changed due to existence of other isotopes of oxygen. When it was decided to use oxygen as reference, it was not known that isotopes of oxygen also exist. Natural oxygen also contains small percentage of other isotopes as well. So, it was found that the atomic mass of 16 actually represents the average of atomic masses of 3 isotopic forms of oxygen. As a result, a second scale was devised in which the atomic mass of principal isotope of oxygen (and not mixture) was taken as 16. This scale, known as physical scale, was preferred by physicists. Thus while chemists used the previous scale, the second scale was being used by physicists. 

In other words, at the same time two different scales were being used, which let to ambiguity in measurements. This led to carbon being chosen as a standard for calculating the atomic masses of different elements. 

2) Hydrogen was used as a reference to determine the atomic masses of elements. Since it is the lightest elements, therefore it  was arbitrarily assigned a mass of 1 (without any units) and other elements were assigned masses relative to it. However, the present system of atomic masses is based on carbon - 12 as the standard and has been agreed upon in 1961.