Preserve building site with pre-fab
If your number one objective is to preserve a building site, the answer for you is to select a pre-manufactured home. The disturbance to the site is limited to building the foundation; and, if you elect not to build a basement, that can be limited to a perimeter foundation and shallow excavation. Pre-manufactured homes also limit the amount of construction waste that is generated during construction.
According to the U.S. Environmental Protection Agency, 2 to 5 pounds of construction waste is generated for every square foot of a home in residential construction. If a home is factory built, raw materials can be ordered and cut to size minimizing the production of waste. Another benefit to factory built homes is that the need to put mechanical systems into service during construction is mitigated so that an owner inherits a mechanical system that has not been exposed to the dust and dirt of construction.
If you don't want to have a home show up on site completed and placed on the foundation, consider using panelized construction for walls, floors and roof construction. While the lead time is longer, the assembly time is very short. These panelized systems have high insulation values and come prepared with conduit and back boxes so all a contractor has to do is pull wire.
But here's a cautionary note: check with your local building code officials to make sure this type of construction is allowable. I was foiled from using this construction method on my own home addition because my town considered these panels in the category of pre-manufactured homes and was prohibited because the inspectors did not like not being able to see how the panels were constructed or the conduit run.
Pre-manufactured does not mean you have to consign yourself to an ugly box. It just means your house is built somewhere else and is brought in some state of completion to your new home site minimizing construction waste, and the disturbance to your site.