Attribution and citation are similar, but not identical. A quick definition:
Attribution is giving credit to the creator of an image or other material in whatever manner that creator has specified. It's their stuff, and you're using it for free, so it's only fair to give credit as requested.
Citation is an integral part of scholarship, in which you name all the sources used in your research. Your sources can include textual information (books, articles, encyclopedia entries) or non-textual material such as images, video clips, statistical tables, etc.
More on attribution vs. citation from open.michigan.
The principle behind citing your sources is intellectual honesty, that is, not claiming someone else's work as your own.
The practice of citation involves using a consistent format to record the information about your sources. Here's a list of guides to various citation styles.
The most well-known category of public domain work is books whose first registration or publication was in the United States before 1923.
What if you want to use material never before published, or published in another country, or in another format such as a sound recording or architectural drawing? Cornell University's Copyright Information Office offers a detailed list.