The different items that you cam use to upgrade your field houses will rot away after different periods of time. The following materials can be used to upgrade:
Grass/ Hay: The weakest
Lumber: Average
Stone: Good
The only place where you can litter is on your field. Anywhere else will cause people to begin disliking you.