Adjusting (aligning) scanned ship plans

[Kris Avonts]

Hello,

 

When using a scanned ship plan as a background image in a CAD program, it often shows some misalignment or even distortion. That makes it more complex to build a precise model out of it.

 

I usually try to fix things with an image manipulation program (e.g. Photoshop, GIMP, …) using several transformations: rotation, scaling, shearing, … and even perspective corrections.

That goes a long way to reach a perfect aligned plan but it is also time consuming and difficult to get it pixel perfect.

 

So I made a little python program that can do a pixel perfect perspective transformation based on set of reference points that you define in the image you want to adjust. Let me show my first attempt with a simple test image.

 

Imagine that this is a sheer plan with a base line at the top of a keel and a fore and aft perpendicular.

The base line is not horizontal and both perpendiculars differ from a vertical with some (different) slant.

 

After transforming it with my program it looks much better.

 

 

The base line is now horizontal and both perpendiculars are vertical. The whole image is remapped using a transformation matrix calculated from the set of given reference points. That is why the edges now show misalignment and slant. The empty pixels at the edges (black) could be corrected by filling the pixels that fall outside the transformation with the background colour (white in this case).

 

At this moment the reference points are hard coded into the program and the test image is also ridiculously small. So it will be of no use to anybody else but I will work and test a bit further until it is more user friendly and then I can upload it here for interested ship plan users.

 

Best regards,

Kris

 

[Egilman]

Currently the only software that directly addresses this is Autodesk's Raster Design.... It is capable of doing exactly what your showing....

 

Hopefully you can develop it into a very useable piece of software....

[Kris Avonts]

It’s time to give some update information and explain how to use the scan-adjust program.

 

I developed this program mainly to fix misalignment and wanted as result a horizontal baseline (top of keel for 18th century ship plans) and vertical perpendiculars. So that is what I call the standard way to adjust a sheer plan. It is illustrated in the drawing below.

You can see how the pixels coordinates are interpreted and where I intend to put the 4 source reference points: As, Bs, Cs and Ds.

   As is at the intersection of the aft perpendicular with the baseline

   Bs is at the intersection of the fore perpendicular with the baseline

   Cs is at the uppermost point of the fore perpendicular

   Ds is at the uppermost point of the aft perpendicular

 

You may observe that these 4 points go round in a counter-clockwise direction and that they delineate a quadrilateral (a figure with 4 corners and 4 sides). The counter-clockwise direction is chosen because it is mentioned in the documentation for the used software (PIL, Python Image Library) that does the image transformation.

 

These 4 source points need to be mapped to 4 destination points. You guessed it: Ad, Bd, Cd and Dd. You can also input these additional points to the program but if the adjustment you are after resembles the ‘standard situation’ then the program can calculate these 4 points based on the knowledge that Bd=Bs, Ad has same y-coordinate as Bd, Cd has same x-coordinate as Bd and Dd has same x-coordinate as Ad. What remains is to find the x-coordinate of Ad and y-coordinates of Cd and Dd. Well there we can make sure that the difference in x-coordinate between Ad and Bd is equal to the number of pixels between As and Bs counted on the line that connects As to Bs. That sounds like trigonometry and it is, see the illustration below.

The calculation of n is used to determine the x-coordinate of Ad. A similar calculation yields y-coordinates for Cd and Dd.

So all destination points are fully defined and the transform can be started.

 

Next I explain how we enter the required data into the program. I choose not to develop a GUI (Graphical User Interface) for my program but simply to look for all required data in a text file that has the same name as the program file but a txt extension. So the program will be ‘scan_adjust.exe’ and the data file ‘scan_adjust.txt’. Next is shown the content of this data file as I used it for my initial test of the program.

 

test-scan.jpg

As=(82, 184)

Bs=(425, 178)

Cs=(429, 48)

Ds=(77, 48)

 

The next problem to tackle is how to get these coordinate values. Well there are very good image manipulation programs with very good GUI’s to zoom into images and by positioning a cursor where you want to set your source point (As, Bs, Cs, or Ds) will show the x and y coordinate values. You can note these down and then enter the values in the scan_adjust.txt file.

 

I always use the GIMP program for this and next I show the steps I take to get these values.

Open your image in GIMP and zoom so that the whole image is shown.

 

Then add 4 horizontal guides (position your cursor in the ruler above the image window, click and drag down) close to the source points you intend to use.

 

Now do the same for 4 vertical guides (position your cursor in the left ruler click and drag into the image)

 

Now you can zoom into each source point and refine the positioning of the guides. Next is shown the final guide positioning for As. I set the horizontal guide above and the vertical left of the reference point.

 

When you position your cursor just below and at the right of the intersection of the guides, you will see the coordinates to use for As at the left in the status bar below the image window (82, 184).

 

Then do the same for Bs, Cs and Ds.

 

Edit the scan_adjust.txt file with your image file name on the first line followed by the 4 source points.

 

How to install my program? Just put it in a folder of your choice. Then add the text file and your image file in the same folder. Double click the scan_adjust.exe file and the transformation will start. It should bring up a CMD window that show some steps that are executed and at the end it waits until you press a key (this is just to help you read its content). The transformed image will have the name of the input image with ‘_a’ added (to refer to the adjusted status) or a file name and extension that you entered as last line in the data file.

When something goes wrong, I try to give a meaningful message and keep the CMD window open until you press a key.

 

Here is an example warning.

 

And her is a successful run.

 

Now I only have to provide you with the program itself. For a more recent or less buggy version see next posts.

scan_adjust.exe

 

Good luck with it and best regards,

    Kris

Edited December 16, 2021 by Kris Avonts
warning for old version added

[Jaager]

This is probably me being impertinent and irreverent.  Consider this as a passing non tenure track faculty member from a different department saying that you may wish to take a step back and look at the problem from a broader perspective..  No selling or any recruitment going on. no insults are intended.

 

Wow!  The plans in post## figure 1 are some seriously messed up plans.

 

My point:  I would be very suspect about how much more accurate any holistic repair of a plan this messed up would be.
There are articles in the old NRJ and Model Shipwright that discuss the distortions from time   on paper or vellum  and from less than perfect camera lenses.
It can be different in different regions of a large plan and a single formula for the whole could be just as unsatisfactory as the original.

As an alternative to a holistic repair:
The data on each station should be considered complete unto itself (until it proves that is not.)
Set up a baseline.
Set the station locations at their ideal intervals.  Isolate each station as an individual.  Rotate each as much as it needs to be vertical.
Take the data points from that to connect the dots. resolve any bad curve runs (fare it).   Fake it for ports, blocks, channels,  steps, etc.  that are between the stations.  i.e - measure the distance from the nearest station.  

I do not do virtual modeling in 3D.  I am dedicated to turning expensive hardwood into sawdust.  Hardcore POF is my area of focus.  I did hand lofting of all frames a few times and decided not to do that any more. Doing that using ink and a drawing board  is nothing if not extremely tedious.  Doing the shapes of 100 plus frames takes a LONG time.    Searching for an alternative,  I could find no 3D CAD that would not allow me to extract every frame should I be able to insert the lines plan.  I am led to believe that a 3d modeler  would allow this - which would then require a whole lot of work rendering a series of cross sections with tight limits.  One for every frame or every bend depending on the framing style.   (NURBS being much better at hard material curves than vertex - which is better for flesh type curves)  (Old memory: Converting a NURBS prop to vertex  version in a vertex based rendering program is like dropping a block of neutronium of a scale.  Way too many points.)

 

I have ultimately found a better way and I only need to use a GIMP clone (Painter 19).  Now,  I do not recommend Painter to anyone else -but I already had it,   having used upgrades of it (Painter 3D) from my Poser 3 days - it does raster, but the bulk of the program is about simulating Bob Ross on a computer screen -- although not much tends to be all that "happy".   A long way of saying that I am an austlander just paying a quick visit.  But we do face similar (the same) problems with less than ideal plans.

[Kris Avonts]

Hello Jaager,

 

I’m very sorry that my post here gave you the impression that this program would solve any issues with misaligned or distorted plans. I wish it could. As for all tools or programs: garbage in is garbage out. The figures I added are more to illustrate the concept then to give a real world example.

It should be used with care and a lot of scepticism regarding the resulting output. That is something I can not code into the program and is left to the judgment of the user.

 

All of the ship plans I looked at until now are plans of the RMG collection. You can find a small fraction (3678 to be precisely) of these plans on the wikimedia commons: https://commons.wikimedia.org/wiki/Category:Ship_plans_of_the_Royal_Museums_Greenwich

 

These plans are scanned and show high resolution but I suppose the alignment on the scanner was done with respect to the edges of the document and not its content. That is one of the reasons to make this program.

 

I will show a real world example to demonstrate what you can expect from the program.

I choose the file ‘Resolution’ (1770) RMG J3351.png and made a cut out of the sheer plan (shown below, reduced to 10% of its original resolution).

 

 

It is a huge file (image size: 20567 x 5521, file size: 25.3MB) and it was also a test if my program could handle it. So I entered the required 4 source points and made a run.

 

 

Apart from giving a warning that the image has more pixels than a preset limit, a result file (image size: 20567 x 5521, file size: 9.25MB) was produced.

 

To show what the differences are, I opened the 2 image files as layers in GIMP. Then by toggling the visibility of the top layer while zoomed in on one of the source points you can observe the changes made by the perspective transformation.

 

This is an animated gif made from 2 screenshots zoomed at point As (intersection of base line with aft perpendicular) before and after the transformation:

 

The same for point Bs (intersection of base line with fore perpendicular):

 

The same for point Cs (top of fore perpendicular):

 

The same for point Ds (top of aft perpendicular):

 

For me this is a reasonable improvement when using such scans as a background image in a 3D CAD program. And even then I never just draw over the lines but make horizontals, verticals and curves (splines) that are fair.

 

Best regards,

Kris

Edited November 13, 2021 by Kris Avonts
fixed the typo Resolution (1700) into Resolution (1770)

[druxey]

Be aware that some distortion is not due to the scanning process, but distortion of the paper itself after hundreds of years of fluctuating humidity in storage. Removing distortion, whether electronically or by manual redrafting is a tedious business!

[Egilman]

7 hours ago, druxey said:

Be aware that some distortion is not due to the scanning process, but distortion of the paper itself after hundreds of years of fluctuating humidity in storage. Removing distortion, whether electronically or by manual redrafting is a tedious business!

And lets not forget the lens distortion, screen aberrations and scanner head tolerances...

 

Realigning a print (squaring) I do in Gimp directly only takes a few minutes and the tools are easy.... removing the distortion is the difficult and tedious part... and in many cases, for modeling purposes, unnecessary....

Edited November 13, 2021 by Egilman

[Jaager]

Kris,

 

Good enough!  you seem to have a realistic and sober perspective on the limitations of your fix.  That is about all that can be expected.  No more worries.

 

I was startled when I saw the quality and style of the Resolution (1700) plan,  because it did not look like any 1700 plan I had seen before. I investigated and it is Resolution (1770) a 74.

A sort of typo that I am very prone to make.   Many of us only wish that there were a lot of plans from around 1700  and beyond any hope that they were that complete.   The mere thought of such is enough to cause ears and eye stalks to go to full alert!  A full on Pavlov response.  😉

 

And as per @Egilman - stacking the layers of frame plans,  shaping and faring a hull,  faring the lines in 3D  all will show what needs to be fixed from distorted plans.  At the time, I sort of doubt that any ship was identical to its design plan.   Wood is forgiving and requires adaptation.   Iron and steel not so much.

 

 

[Kris Avonts]

Thanks Jaager for spotting my typo. I corrected it before anyone else starts a search for that ship.

 

[Kris Avonts]

Hello again,

 

While the result of the sheer plan adjustment with my scan_adjust.exe program was not bad, I noticed that it still has some deviations in the mid-ship area. The top of keel line on the plan was a little above the horizontal guide in GIMP that connects the intersections of fore and aft perpendiculars with the base line. Also some residual slant could be noticed for station (3). I wondered if that could be corrected using the same program.

 

 

It is possible but it takes several steps:

• Divide the image in 2 halves with station (3) as the separating edge but allow some overlap (80 pixels)
• I named these J3351_sheer_a_aft.jpg and J3351_sheer_a_fore.jpg
• J3351_sheer_a_aft was adjusted with both source and destination points given: As = Ad, Ds = Dd, Bd 6 pixels below Bs and Cd 6 pixels below and 8 pixels right of Cs
• J3351_sheer_a_fore was adjusted with both source and destination points given: Bs = Bd, Cs = Cd, Ad 6 pixels below As and Dd 6 pixels below and 8 pixels right of Ds
• Then the 2 resulting half images and the original full image are opened as layers in GIMP.
• Shift the 2 half images until they stitch nicely (Bd and Cd of the aft half should map on Ad and Dd of the fore half)
• The layer on top (fore half in my case) will spoil the stitching because its edge has black (not transformed) pixels, so make a crop to get rid of these pixels
• Remove the layer with the full image and merge down what is left over

That should give you a result with almost perfectly horizontal and flat base line and vertical stations.

 

 

I do realise that this is a quite complex process just to correct for a mere 6 or 8 pixels. When I find some time, I may upgrade my scan_adjust program to recognise this special case and do all the required steps in the background and just saves the final result.

 

Best regards,

Kris

[Kris Avonts]

Hello,

 

It's time for an update.

The past weeks I worked further on this program and added several new features that can make it easier to do a transformation.

 

In my previous post I promised to include a workflow to do a 'flattening' action for the base line. The several steps described in that post are now replaced by a single mesh transform. That is a transformation that maps a series of quadrilaterals onto a series of rectangles. This solves the problem of stitching the different parts that was present as the last step in my original workflow.

 

I will not go into every detail for using this new version of the program here because I have made a user manual.

ship-plan-adjuster_UM.pdf

 

The name of the program was changed from scan-adjust into ship-plan-adjuster because that is just what I expect it to do.

 

The most important addition is the 'warp' transformation that can do most of the work in one single step.

The next figure illustrates what it is expected to do.

For all details about this transformation, the points to input and how to use the program, please read the user manual.

And here is the new program. See next posts for a more recent or less buggy version.

ship-plan-adjuster.exe

 

Although there was not much reaction on this topic I noticed it had quite some downloads.

So maybe some of you can make use of it.

If it serves your needs then it would be nice if you could show some examples here.

Thanks in advance.

 

As a last note I must warn that this program can not solve every issue that you may want to fix in a scanned ship plan.

So always check the output image and reject it when it doesn't match your expectations.

 

best regards,

   Kris

Edited December 16, 2021 by Kris Avonts

[druxey]

I've not downloaded or tried this program yet, but it looks quite ingenious and very useful! Thank you for making it available, Kris.

[Kris Avonts]

Thanks druxey for your trust in my programming skills.

 

I must however admit that some ship plans are a bit to complex to benefit from the auto distribution calculation for the stations.

During the development I also tried the sheer view of the Hayling Hoy, but that was no success. I guess it was not designed according a 'room and space' rule. I know it is not a Royal Navy vessel.

In such cases you are on your own. But with the help of a spreadsheet program you could check and eventually correct station distances by entering the optional destination points Xd.

 

If there is some interest in what happens behind the scenes in the program, I could give some more information showing also the (simple) mathematics that come to the rescue.

 

best regards,

   Kris

[Kris Avonts]

Just some addition about the transforms that are used.

 

The first step is always a perspective transform and that does exactly what its name hints. It corrects the perspective view in 2D of a given object. It is especially useful for aligning features in an image of which you know that they should be vertical or horizontal.

I made the following illustration to show how it is used to get the base line of a ship plan horizontal and the perpendiculars vertical.

The transform squeezes the blue quadrilateral (figure with 4 sides and 4 corners) into the red rectangle.

As you can see at the edges, the blue quadrilateral extends beyond the original ship plan image (black rectangle). That will cause points in the red rectangle that will not get a transformed pixel. These will get a fill colour that will be close to the average background colour.

 

The next problem to tackle is flattening the base line and setting some stations vertical. That is a perfect task for a mesh transform. In the Pillow image manipulation library it is described as a series of quad transforms that are done one after the other on the same image.

Now we must explain a quad transform, .... well that is just a transform which again maps a quadrilateral into a rectangle.

Lets make it more clear with a next illustration.

Here you see all the different quad transforms numbered from 1 to 6 that take place for a flattening action when Es and Fs each have 3 coordinate values. We start at the left side and map a blue rectangle with the aft perpendicular as its right side to the red rectangle that has the same size. This is actually a copy. That part was already corrected by the previous transform.

Then the real work starts: the next quadrilateral has a left side that is taken over from the one to its left, the right side is a line passing through an Es, Fs pair, the top and bottom sides are parallel with a line through A and Es[0] (or Es[0] and Es[1], Es[1] and Es[2] ....you get the point). It is transformed to the red rectangle with its right side passing through Es[0] (or Es[1], Es[2] ...).

The same action is repeated until we reach the fore perpendicular. The remaining part is again a simple copy.

Then we are done

 

The warp transform is a sequence of a perspective transform followed by a mesh transform with the one exception that the right sides of the red rectangles will not be defined by the Ls points (same as Es shown in the illustration above) but by distributed x coordinates Xd that are calculated by the program or given as extra input by the user.

 

I did not include any math here to keep the description as intuitive as possible.

 

For those who want a more closer look to the illustrations, I provide here also the pdf version where you can zoom for details.

perspective-transform.pdfmesh-transform.pdf

 

That is it for now. I hope that takes away a bit of the hocus pocus by just clicking an exe program.

Best regards,

   Kris

 

[Kris Avonts]

Sorry guys, I have to report a bug in the program.

 

When doing a distribute or warp action on an image and you do give the optional x coordinates Xd in the ship-plan-adjuster.txt file, then the program crashes. You can see that by the CMD window suddenly closing unexpectedly. There will be no output image file nor text file.

 

I ran into this when I tried to warp a half-breadth view and reused the Xd data that was calculated in a previous warping of the sheer view.

I already corrected the program and changed the version it reports to 0.3 (mentioned on the first line in the CMD window or the output text file).

 

Here is the update:

ship-plan-adjuster.exe

 

Hopefully it didn't cause much frustration yet.

 

best regards,

   Kris

[Waldemar]

Hello Kris,

 

I have been looking for such plan adjuster for years (rather unsuccessfully), and have given yours a try. The source bitmap drawing I have used is distorted in so many ways, that it was originally almost impossible to redraw it credibly, even manually, into the vector graphic.

 

Original plan:

 

 

Only basic transformations were applied so far (i.e. squaring and flattening), giving already outstanding results, simply impossible to achieve in common bitmap editors. As it was just an exercise, the drawing with all its views was adjusted as a whole (which should be not of course) by your program, taking as a reference half-breadth rectangle.

 

Here’s the outcome:

 

 

 

What graphic formats does your program support except PNG and JPG?

 

Great job. Many thanks for this.

 

Waldemar

[Dart]

[Kris Avonts]

Hello Waldemar,

 

Thanks for sharing your results with my transformation program.

It can be a help before you start CAD work.

 

My program uses the 'Python Image Library' also known as 'Pillow'.

It supports a large amount of image file formats: BMP, GIF, JPEG, PNG, TIFF, ...

You can find more information here: https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html

I did not try all these formats but only JPEG and PNG.

 

@Dart, maybe you can explain in some words how you tackled the problem.

 

best regards,

   Kris

[Waldemar]

Actually, I did an attempt of 3D reconstruction of this very plan some time ago too (with deliberate, slight modifications to the ship’s lines). Ultimately possible in many cases, but it is usually a pain to work with distorted drawings. Besides, distorted plans take considerably more time to interpret them and the accuracy is jeopardized.

 

 

I wish I could use your program before, in this project and in some other as well. Your ship plan adjuster just do what was desperately needed. But now it is ready for the next projects, which makes me very optimistic... Many thanks again for sharing it.

 

Warmest greetings,
Waldemar

Edited January 17, 2022 by Waldemar

[Ben752]

Just want to point out this is:

1. Awesome
2. Computer Vision research

 

The known references certainly assist dramatically.  I recall reading a paper about removing lense distortions by using a reference photo of a checkerboard, fitting the edges to make grid points and then fitting a transform function.  Seems like an adaptable solution. 

https://mzucker.github.io/2016/08/15/page-dewarping.html

https://arxiv.org/pdf/2007.09824.pdf

https://github.com/gwxie/Dewarping-Document-Image-By-Displacement-Flow-Estimation/blob/main/90_paper.pdf

[druxey]

Ben: your grid method will only partially solve the problem. It will certainly deal with lens distortion, but does nothing to remove paper or printing distortion. 

[Kris Avonts]

Hi Ben,

 

Thank you for the appreciation of my image transformation program. It is however less advanced than you might think. I like to keep things as simple as possible but still try to reach the target I have in mind.

A checkerboard correction method would only be applicable in the sheer view where you have the waterlines that should be straight and equidistant. The complexity of the transformation would become much higher and have no effect on the half-breadth plan where the waterlines are most important to give a representation of the ship's lines.

Only when you could combine the intersection points of waterlines with station-planes of all 3 views (that is a lot of points) to map exactly on the same 3D point (most often called a vertex) by a transformation it makes sense. I think that is best solved in a 3D modelling program.

 

@druxey, the ship plan scans of the RMG are such high quality, I think there is very little lens distortion. The paper distortion is more a concern and very difficult to correct due its irregular nature.

 

I use my own transformation program to get the 3 views out of a ship plan and have them properly aligned before importing into a 3D drawing program.

The workflow to make the transformations is important as I noticed in this detail of a half-breadth plan.

 

I always try to analyse what could cause such artifacts. In this case it is not a bug in the transformation program but just me wanting to do correction of the half-breadth plan in just one go (single warp transform) instead of a 2 step square and warp transform.

So you need to plan carefully the steps to follow, certainly when reusing results from previous transforms in other views (sheer in this case).

 

best regards,

   Kris

 

 

 

[druxey]

I was assuming the writer was referring to photographed rather than flat-scanned images, Kris.

 

Either way, it's a complex task to correct, whether by CAD program or on a drafting board!

[Ben752]

Kris,

 

I shared a few things just to show some similar research but totally agree with your analysis.  

 

I was curious if you happen to have Mac version or even better a GitHub repo with the source?  I’ve encountered the same challenge and love your approach.

As an aside, I wrote a small a Python script that used scipy’s genetic algorithm to fit splines that output control points compatible with how Fusion 360 constructed knots.

Great work!

 

Edited February 12, 2022 by Ben752

[Kris Avonts]

Hi Ben,

 

For you and anyone interested I can share the python source code. There are some comments written in the code file that can explain the order of things and the calculations done.

If you encounter difficulties or other unclear parts in the code you can always send me a personal mail.

This is the code file:

ship-plan-adjuster.py

 

best regards,

   Kris