Not all banks from before the civil war went out of business. The Farmers Bank of Mount Holly, NJ is an example. It became the Farmers' National Bank of New Jersey at Mount Holly in 1865 and eventually the Farmer's Trust Company in 1912 Source: New Jersey's Money by George W. Wait (1976).
6.25 cents, Farmer's Bank of Mount Holly, NJ (1815)
$10 Farmer's Bank of Mount Holly, NJ (1849)