The first thing to consider is what kind of pet you have. The cheapest kind of pet insurance is that for the most common pets - cats and dogs. Prices for these policies can be very reasonable indeed, amounting to only the cost of a few tins of pet food a month. If, however, you keep a more unusual kind of animal as a pet, then the price you pay may well be higher. Make sure that any policy you take out specifically includes your kind of pet, as many will exclude more 'exotic' kinds of animal.
The main reason for considering pet insurance is cover for medical treatment, but there are some exclusions that you need to be aware of. Firstly, your policy will almost certainly not cover routine treatments such as vaccinations, flea control, or worming. Secondly, you won't be covered for any pre-existing conditions that were already known about when you took out the policy (you should also declare any existing conditions when you take out the policy, or you risk it being declared invalid when you come to make a claim).
You also need to check your policy for claim limits: there may be a time limit of cover, so for example a long-term condition such as arthritis may only be covered for a period of 12 months, after which you will be responsible for meeting the bills. There is also usually a limit to the total cost of medical bill claims you make in any one year, but this is usually high enough to cover almost any conceivable situation.
Another major reason for taking out insurance is to help you cover the costs of getting your pet back if it goes missing. Most policies will contribute towards advertising in local newspapers etcetera, and also to providing reward money for the safe return of your pet.
Another very important part of your insurance cover is public liability, especially for dog owners. Even the calmest and most well behaved of pets could possibly cause damage to someone's property, or even cause an injury to a person. In this thankfully rare event, you could be open to huge legal bills and compensation claims - make sure that your policy includes a substantial amount of third party liability cover as standard.
Finally, check whether the policy you're considering places a limit on the age of your pet. Many policies are only available to, for example, cats up until the age of eight years. As it is in later years that your pet is most likely to need treatment, it obviously makes sense to ensure your policy will cover this, especially if your pet has a few years under their belt!