They touch the house because they didn't want their mother to a laughing stock as they would need a crane to get her out. They (johny Depp character) decided to touch the house and burn thier connection to their mother and thier past. They are free from all the shit that they had to deal with in that shitty town.