When you are trying to decide whether you can afford to refinance you must look at several variables that determine how affordable it really is.  While some people may like the idea of refinancing it may not be very practical.  There are several important things you should consider when you are looking into refinancing.  

First, you need to look at the interest rate you are paying now, and what you could potentially have for an interest rate if you refinanced.  Remember, your interest rate is what determines how much of your payment is allotted towards interest only on the mortgage.  The lower your interest rate the better off you are and the faster you are able to pay the principal.  Because of this reasoning, it would be impractical to refinance your mortgage to have a higher interest rate.  

Second, would be to change the financing type. E.g. if you have a jumbo or even an ARM or adjustable rate mortgage, you may want to refinance in order to change to a fixed rate. 
This is often a smart move if you think the interest rates are going to be rising and cannot afford to take a chance on what the potential interest rates could become.  It is a very smart move to change to a fixed mortgage if you suspect the higher rates are coming, however switching to an ARM might be wise if you sense a huge drop in the rates over a period of time.  

Third is that you may desire changing the term of the loan.  For example if you started with a 30-year loan, you may wish to change to a 20 or even 15 year mortgage.  Alternatively, if you are looking to reduce your payments you could change to a longer term than what you currently have.  The reason depends on your exact needs financially.  Some people are able to lose a few years off their mortgage because of drops in interest rates.  If the rates plummet almost 50% off what you are currently paying, you can afford to reduce the term of your mortgage by approximately half without increasing your payments by much if any at all.  This can reduce your overall paid price for your home and still have your home paid off much sooner.

Fourth, you may need equity from the home in order to pay for something such as a remodel, college education, retirement, vacation, or even a major health care expense.  The possibilities here are endless, but many times a refinance is chosen because of the ability to cash out equity.  However, you need to ensure before choosing this option that you can get enough equity out to be worth the fees and additional years added to your mortgage.  

Finally, you may want to refinance in order to remove the PMI or private mortgage insurance from your mortgage.  The PMI payment is often quite expensive and being able to remove it from your mortgage can equal huge savings each month and even over the life of the mortgage.  You should not refinance your mortgage to remove the PMI unless you have at least 20% equity in the home.  Otherwise, you are just ensuring that once again you will need to purchase PMI for the mortgage.  

As you can see there are several good reasons to refinance, however you should compare the advantages to the costs associated with a refinance in order to ensure that overall it is a good decision financially.  If the financial outcome is questionable, you might want to postpone a refinance.