To amend the Internal Revenue Code of 1986 to impose an excise tax on the international transportation of individuals by water, to establish the Caribbean Ports and Infrastructure Improvement Trust Fund and the Water and Marine Wildlife Protection Trust Fund, and for other purposes.

Pay Back America Act of 2004 - Amends the Internal Revenue Code to impose a per passenger tax on covered international voyages of commercial passenger vessels. Establishes the Caribbean Ports and Infrastructure Protection Trust Fund (Caribbean Fund) and the Water and Marine Wildlife Protection Trust Fund (Wildlife Fund) in the Treasury. Appropriates to those funds a percentage of the...