4 hours should more than enough to get from LGA to the port. Assuming that your plane and luggage all come out on schedule, you should be at the port by 3pm.
I don't know about buses, but here is a website that should give you some info:
This is the NYC mass transit site.
For a cab, it should not cost more than $30 (+ $3.50 for toll + tip). Of course, if there is a lot of traffic this will affect the price, but on a Saturday afternoon you should be fine. It will take about 40 - 45 minutes to get to the port from LGA. It is only about 12 miles away, but the problem is that you have to drive through Manhattan, and going cross-town takes longer.
Have a great cruise!