A backup file may contain information like database dumps or source code files that may be used by a malicious attacker to gain a deep understanding of the application structure and contents.
You should delete backups files or move them into a directory which is not publicly accessible.
Backup files may be left behind by automated backup software. Temporary files are sometimes also created by various text editors. Common extensions for backup files are: .bkp, .old, .backup, .swp and .ext~