This post may be long, but you asked for it....
So first question: If ocean liners ply only the Atlantic (a fairly rough patch of water), why would you call for a less stable ocean liner hull? Fuel economy, perhaps -- but at the cost of seasick passengers? A cruise ship hull would seem more suitable.
Okay, time for an experiment. Rinse out an empty tuna (or similar) can and float it in seveal inches of water in your bathtub. You'll see that it's nice and stable because it's much wider than its draft.
Now, plunge your forearm up and down in the other end of the tub to make some fairly big waves and watch what the tin can does. You'll see it bob up and down and tilt side to side (that is, roll) as waves pass by its location.
The problem with the tin can is not one of instability, but rather one of too much stability, which causes the tuna can to conform to every passing wave. Basically, the tuna can rolls pretty seerely because it reacts very quickly to changes in slope in the surface of the water.
If you want to carry this expriment one step further, you can put a few small weights in the bottom of a vegetable can, so about its radius is below the waterline, for comparison. The weighted soup can won't respond as quickly to the wave actoin, and thus will roll less than the tuna can.
Ships face a similar problem, not so much with the short waves that are most noticeable but rather with the longer (typically hundreds of feet crest to crest) swells that your eye can pick out by looking at the crests of the smaller waves. In order to make a liner ride smootnly without stabilizers, one must design the hull to be considerably less stable than the tuna can. The "deep V" hull meets this requirement. The diminished stability causes the vessel to respond very slowly to changes in slope, so the rise of the swell passes and the fall of the swell begins to counter it well before the vessel fully responds to the rise.
Note, BTW, that I did NOT say that an ocean liner should be unstable! An unstable vessel would not remain on even keel even in calm waters.
You can also see this by comparing a houseboat (shallow, very stable, hull) to a cabin cruiser (less stable deep "V" hull). You'll feel a lot less motion on the cabin cruiser if there are waves of any size -- which is why most people don't take houseboats out to open waters.
Second question: Battleship hull design. Every battleship that I'm aware of had an ocean liner type hull. And yet, a battleship is basically a gun platform. And as such, you want the most stable hull design possible. If the ship rocks too much from the recoil of the main battery, precious time is lost until the main battery bears upon the target.
There are three parts to the answer to this.
>> 1. Battleships do indeed have deep hulls, similar to ocean liners, to diminish their susceptibility to heavy seas.
>> 2. Battleships have a LOT of armor plate, and thus a LOT of mass and inertia, for their size, which redues the amount of motion imparted by a given force or torque.
>> 3. The guns on all World War II vintage and newer warships are dynamically stabilized. Their control systems actually sense the motion of the hull and compensate for it, so the guns stay locked on their targets. It's really impressive to see this in action on a ship that's rolling twenty or thirty degrees!
Interestingly, the German battleship Bismarck (and its sister Tirpitz) had ocean liner type hulls, but had a beam of 36 meters! The Bismarck was regarded as one of the most stable battleship gun platforms ever built. But wouldn't it have been even more stable with a cruise ship type hull? Of course it's range would have been reduced, but I suppose everything is a compromise.
Stable in what sense? A platform that's too stable, by the definition of system dynamics, will roll a lot more severely than one that isn't. Unfortunately, laymen tend to use the term "stable" to mean "not susceptible to roll"....
Just thought I'd pick your brain a little!
I hope that this answers your questions!