What kind of technical support do you have?
If the answer is anything less than 24 hours, seven days a week, find another provider.

Will I have room to grow?
If you need only Internet access now, but expect to develop a web site fairly soon, make sure that the ISP has the facilities to host the size of site you expect to have.

Furthermore, if you think you'll need help designing your web site, find out if they have any designers on staff - or the resources to find you one.

Can I cancel my contract at any time?
You don't want to get stuck paying for access that you aren't happy with.

Is the client roster similar to your business?
You don't want to be lost in a sea of clients whose business is more important than yours to the ISP. Equally, the ISP should be able to handle the bandwidth you need - and the bandwidth you may need as you grow.

Can I talk to references?
Any ISP worth using should be able to give you at least three references -- again, businesses that are similar to yours in terms of what they need from the ISP.

What do you think I need?
Even if you have a good idea of what you need - especially if you have a good idea of what you need - find out what the provider thinks you should have. Let them ask you questions and arrive at their own conclusions, and see how it meshes with what others have suggested or what you have concluded yourself.

Do you have a high-speed direct connection to the Internet?
This is different from the connection you will use to connect to the ISP. The ISP itself should connect to the Internet at a very high speed - with a leased line from a telecommunications company, for example.