If you use the delete project option. Two tick boxes are availeble.
1) Delete media (this ticked will remove all related media from your system)
2) Delete render files (which will delete all render files)
After this no files should be remaining. An exception could be if you import media with the LINK option. As those files become not really part of the project, and only the shortcut (LINK) to the media will be removed.