I have a trained model to test images of 256x256 size only. But I have to test very big images of higher resolution, for example, 2048x2048.
This can be done while virtually dividing the image into non-overlapping patches of 256x256 and then predicting them by one by one, and then stitching them back to the full image, and then have to calculate the metrics comparing original and final ...
I am not able to make python code for this. Can somebody help
Related
I am building a cycleGAN with a u-net structure to improve the image quality of Cone Beam Computer Tomography (CBCT), with the Fan Beam Computer Tomography FBCT) being the target images. Because I want to mainly enhance the quality in the lung region, I crop the lung volumes out of the original images and assign value 0 to the remaining regions other than the lung (given that the lung pixel value ranges from -1000 to 0). Scaling and normalization are done to the array with the range of -1000 to 0. Please see the following images as an example for my dataset:
In the example above, the left most image is the CBCT and the right most image is the FBCT. The middle one is the generated image from the CBCT by the cycleGAN model. This is actually an example from the very early epoch of the training.
But when the model goes on training, it somehow gradually loses its ability to capture the anatomy of the images, and eventually generate a blank image with all value 0. (see image below)
The loss along the training is climbing back up after several epoch and it starts to lose the anatomy information:
What makes me curious is that such loss of anatomy does not occur when I simply input the whole CBCT and FBCT images as the dataset, without doing lung segmentation nor value assignment to the regions outside the lung. If un-segmented images are given, the model actually successfully translate the CBCT into mimicking the FBCT quality. I do the segmentation since I want the model to only concentrate in the lung region to see if it performs better.
I wonder if this is the consequences with that the background has extremely high value than the region of interest (i.e. background value: 0; lung value: -1000 to 0). Is there any work published on cycleGAN training with images containing blank background? If yes, is there any special measure when assigning value to the background, or when doing normalization and scaling? I can’t really find any so far.
Any insight is appreciated. Thank you.
I have 2 medical image datasets for a given patient, each acquired at the same time, each with a different modality. The frame of reference or coordinate space is different for each dataset (and I don't know the origins). One dataset has smaller physical dimensions than the other, the voxel sizes, as well as the number of frames are also different. I want to resample and register the images, does it matter which I do first?
It's better if you resize the images first. This is because image registration algorithms will try to find a transformation that optimizes the match of the images. If you know the proper size of the images, it will be easier for the algorithms to find such transformation.
I am new to the field of medical imaging - and trying to solve this (potentially basic problem). For a machine learning purpose, I am trying to standardize and normalize a library of DICOM images, to ensure that all images have the same rotation and are at the same scale (e.g. in mm). I have been playing around with the Mango viewer, and understand that one can create transformation matrices that might be helpful in this regard. I have however the following basic questions:
I would have thought that a scaling of the image would have changed the pixel spacing in the image header. Does this tag not provide the distance between pixels, and should this not change as a result of scaling?
What is the easiest way to standardize a library of images (ideally in python)? Is it possible and should one extract a mean pixel spacing across all images, and then scaling all images to match that mean? or is there a smarter way to ensure consistency in scaling and rotation?
Many thanks in advance, W
Does this tag not provide the distance between pixels, and should this
not change as a result of scaling?
Think of the image voxels as fixed units of space, which are sampling your image. When you apply your transform, you are translating/rotating/scaling your image around within these fixed units of space. That is, the size and shape of the voxels doesn't change. They just sample different parts of your image.
You can resample your image by making your voxels bigger or smaller or changing their shape (pixel spacing), but this can be independent of the transform you are applying to the image.
What is the easiest way to standardize a library of images (ideally in
python)?
One option is FSL-FLIRT, although it only accepts data in NIFTI format, so you'd have to convert your DICOMs to NIFTI. There is also this Python interface to FSL.
Is it possible and should one extract a mean pixel spacing across all
images, and then scaling all images to match that mean? or is there a
smarter way to ensure consistency in scaling and rotation?
I think you'd just to have pick a reference image to register all your other images too. There's no right answer: picking the highest resolution image/voxel dimensions or an average or some resampling into some other set of dimensions all sound reasonable.
Suppose we have formed the codebook using the RGB training images. This codebook is now present at the encoder and the decoder.
Now we have one RGB test image (which is not contained in the training images) which we want to compress, transmit and reconstruct. During the reconstruction of this test image, due to the different intensities of the test image, some of which might completely not match with any training image intensity, wouldn't parts of the reconstructed image be darker or brighter than the original image in existing vector quantization algorithms? Is there any way deal with the intensities in existing algorithms like K-means, LBG? Or should we make an appropriate choice of the training images to begin with? Or should the test image be also included within the training images? What is the standard way?
Vector Quantization is a lossy compression scheme. You are finding the best-match clusters within the training set to create the codebook. It is an approximation. The larger the training set, the better the match will be, but there will always be loss.
Your training set needs to account for all intensities (complexities) of images, not only the intensity of the image you intend to compress. Whether or not the training images contain the test image won't change the fact that loss will occur (any gain will be insignificant, unless the training set is very small).
I'm not sure this is the right place but here I go:
I have a database of 300 picture in high-resolution. I want to compute the PCA on this database and so far here is what I do: - reshape every image as a single column vector - create a matrix of all my data (500x300) - compute the average column and substract it to my matrix, this gives me X - compute the correlation C = X'X (300x300) - find the eigenvectors V and Eigen Values D of C. - the PCA matrix is given by XV*D^-1/2, where each column is a Principal Component
This is great and gives me correct component.
Now what I'm doing is doing the same PCA on the same database, except that the images have a lower resolution.
Here are my results, low-res on the left and high-res on the right. Has you can see most of them are similar but SOME images are not the same (the ones I circled)
Is there any way to explain this? I need for my algorithm to have the same images, but one set in high-res and the other one in low-res, how can I make this happen?
thanks
It is very possible that the filter you used could have done a thing or two to some of the components. After all, lower resolution images don't contain higher frequencies that, too, contribute to which components you're going to get. If component weights (lambdas) at those images are small, there's also a good possibility of errors.
I'm guessing your component images are sorted by weight. If they are, I would try to use a different pre-downsampling filter and see if it gives different results (essentially obtain lower resolution images by different means). It is possible that the components that come out differently have lots of frequency content in the transition band of that filter. It looks like images circled with red are nearly perfect inversions of each other. Filters can cause such things.
If your images are not sorted by weight, I wouldn't be surprised if the ones you circled have very little weight and that could simply be a computational precision error or something of that sort. In any case, we would probably need a little more information about how you downsample, how you sort the images before displaying them. Also, I wouldn't expect all images to be extremely similar because you're essentially getting rid of quite a few frequency components. I'm pretty sure it wouldn't have anything to do with the fact that you're stretching out images into vectors to compute PCA, but try to stretch them out in a different direction (take columns instead of rows or vice versa) and try that. If it changes the result, then perhaps you might want to try to perform PCA somewhat differently, not sure how.