Yes, the FAR and supps can be overwhelming! Responses to your questions are in bold font below.

     -Per 25.003- definition of Construction Material, “Materials purchased directly by the Government are supplies, not construction material.”  Am I correct to consider material purchased by the Government for use by military labor construction is a supply and not a construction material?  Yes. The phrases "...brought to the construction site by a contractor or subcontractor for incorporation into the building or work" and "Materials purchased directly by the Government are supplies, not construction material" provide a pair of fairly clear criteria to distinguish between the two categories.
     -Per 25.003- applicability of subparts chart, neither 25.1 “Buy American Supplies” or 25.2 “Buy American Construction Materials” applies outside US.  Does this mean that the Buy-American Act does not apply to overseas construction projects built with military labor? Correct
     -Does it matter if they are to be purchased via Defense Logistics Agency or other provider? This answer is found in the answer to the first question.  -Will this vary depending on location (by country, on/off base, military exercise, etc)? No
  -Will this vary depending if it is raw materials (ex: lumber, crushed stone) or manufactured materials (ex: geotextiles, electrical boxes, acoustic ceiling) No
     -Are there any else that I have not considered? Nothing I can think of. You may have been trying to look a bit too deeply into the definition for construction material.