-
Notifications
You must be signed in to change notification settings - Fork 94
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Trying to train SynSin on SceneNet database #24
Comments
This is the command I'm using to start the training
I wrote DataLoader for SceneNet based on KittiDataLoader. The code is as follows:
|
I think it's probably something with the camera set up -- you should see when it first projects stuff that the noisy results somewhat align with the true images. You can try using the true depths in the code in order to see if the cameras are right (here: https://github.com/facebookresearch/synsin/blob/master/models/z_buffermodel.py#L89). |
Thanks @oawiles. I'm using true depth only. I'll check if warping of features is correct. |
You can also try warping the RGB -- e.g. pass the RGB colours as features. This should be easier to check. Then these should precisely match the other image. |
This comment has been minimized.
This comment has been minimized.
@oawiles, you were right. The error is during warping only. The output of splatter is just an array of zeros. The error is in the format of camera matrix. By writing my own transformation code, I'm able to train the SynSin model. But, I'm not able to get your transformation (warping) code to work correctly. I had the camera matrix in the form With this camera matrix, splatter output was zeros. I changed the camera matrix and removed dependencies on height and width of frame as follows With this, splatter output is a warped frame, but the transformation doesn't match with the ground truth. Can you suggest what changes I've to make to my camera matrix? In other words, in what format does your code camera matrix to be in? Thanks a lot |
What is the error? Sometimes how the splattered image looks in comparison to the true image makes it make snese. One thing I notice is that you should use K to make the values between -1,1 which I believe is not what you're doing. Another thing is sometimes you have to flip the Y. Without being able to see the visual results it's hard to guess at the precise problem. |
It could make a difference. I would recommend you first try to resize. Otherwise I think the intrinsics would mess it up. It l ooks like it's zoomed in, which could be from the cropping. I'd recommend first resizing and then using a matrix to transform from the intrinsics to [-1,1] for x/y using an offset matrix O such that you have a new intrinsic matrix I = O K where K was your old intrinsic matrix. |
OK. I'll try that. Thanks! |
Hi I have similar issues as described in the first message of this thread. I'm trying to train the code on my own dataset. I do save out the warped images using gt depth with the 'use_rgb_features' option set to True and they do look good. However, the model doesn't really train and I continue to get images that are mostly a single color. I tried debugging with only using L1 loss etc. but I observe the same pattern. Do you have any other pointers to what could be the issue? |
Hi,
I'm trying to train the SynSin model on SceneNet database. But I'm not able to train the model. I would really appreciate it, if you can give me some tips.
--use_gt_depth
flag.D_Real
andD_Fake
have similar values in each batch (around 0.1 to 0.3). So, discriminator isn't training as well.I don't know what else to try. Can you kindly help me out here?
The text was updated successfully, but these errors were encountered: