Ivy is sortof like a parasite to a brick house. It will actually take the mortar OUT of the cracks between the bricks making the "lifetime" of the brick and mortar much much shorter than a house without the ivy.
In fact, just recently, there was an old house down the road from here that was very beautiful, large, and brick. Well, the old owner passed on and a new couple bought it. In further exploration of the brick and mortar (after the first winter when they found the house hard to heat) They found that the Ivy (which was covering nearly all of two sides of the house) was actually creating large holes in the mortar.
Due to this, unless you are doing it on a brick wall that is not attached to your house (like a brick "fence" around your property) I do not ever recommend letting Ivy grow on the walls...