The hidden web
Searching the internet is not as easy as just doing a search in Google.
A significant amount of information on the web is unindexed and cannot be found through standard search engines. This is known as the deep (hidden or invisible) web.
This is not the same as the dark web, used by hackers and criminals.
This hidden infromation includes:
- Non-HTML files (PDF files, etc.)
- Databases or archives within websites
- Sites requiring registration or login
- Dynamically created Web pages
- Interactive tools (calculators, etc.)
Image: Wikimedia Commons/Ap470 CC
Hints for accessing hidden information
- Use research clearinghouses!
- Put the word database or archive in your Google query eg. megafauna database
- You will get a list of spaces that you can then search within, not the papers itself
- Check websites of key organisations, government departments and institutions
- Academic profiles can contain information and llinks to unpublished works
- Use a deep web search engine
Research clearing houses or directories are web spaces created by institutions or organisations to compile information on their area of interest. These repositories contain a vast wealth of materials and are key to searching grey literature.
Do a search for your broad topic and the word clearinghouse in Google. eg archaeology clearinghouse
Deep Web Search Engines
A small selection of available resources: