Why I'm using DeepSNR For Noise Removal and You Can Too

Рет қаралды 8,994

Ай бұрын

Sorry Topaz, there is a new sheriff in town for AI astrophotography noise removal. Download DeepSNR from www.deepsnrastro.com
And thanks to the show sponsor, the legends at High Point Scientific!
HPS Website : bit.ly/3oW0PMx
Support the channel by buying Dylan's Telescopes, Cameras & Equipment using the links below!
EQUIPMENT LIST
----
Celestron 11" Edge HD Telescope
USA : bit.ly/3JB95JE
Apertura All Night Imaging Power Supply - 518Wh Lithium
bit.ly/4ctkzhF
QHY200M Planetary Camera:
bit.ly/4152EbY
Celestron RASA F2 11" Telescope
USA : bit.ly/3Jy4Ezl
International : amzn.to/3ABsVBz
QHY268M Mono CMOS Camera
USA : bit.ly/33u4k5w
Skywatcher EQ8Rh Pro Mount
USA : bit.ly/3gTdWJX
AUS : bit.ly/3dxdRv1
Sky-Watcher Star Adventurer GTI Tracking Mount
USA : bit.ly/3sUFOXZ
International : amzn.to/3ABLAwQ
SkyWatcher EVOLUX Telescopes :
USA : bit.ly/36henwd
NexDome Observatory by Sidereal Trading (Australia)
www.siderealtrading.com.au
Celestron RASA 8" F2 Telescope
USA : bit.ly/3GTUwiU
International : amzn.to/3G4nlbP
Celestron NexYZ 3-Axis Smartphone Adapter
USA : bit.ly/351K2ki
International : amzn.to/3g20wLd
Celestron CGX Mount
USA : bit.ly/3BtPVCF
Lunt40mm Solar Telescope
USA : bit.ly/3sP8KMX
ZWO ASI 174MM (Planetary / Solar Camera)
USA : bit.ly/3uXpEM8
International : amzn.to/3rXqJAj
ZWO ASI 120MM (Guide Camera)
USA : bit.ly/352iBXg
ZWO ASI 1600MM (Deep Space Camera)
USA : bit.ly/3HVtvg1
International : amzn.to/3rXqJAj
Celestron 0.7x reducer for 11" Edge HD
USA : bit.ly/3gTDJSy
Celestron 0.7x reducer for 9.25” Edge HD
USA : bit.ly/3sNyY2o
Celestron 9.25” Edge HD
USA : bit.ly/3LISupw
Canon 6D mkII DSLR
International : amzn.to/2CbiVER
Tokina 16-28mm Lens
International : amzn.to/3fdFJCo
Canon EF 16-35mm f/2.8L III USM Ultra Wide Angle Zoom Full Frame
International : amzn.to/3G9kjmP
Canon EF 100-400mm f/4.5-5.6L IS II USM Lens
International : amzn.to/3H9yPfA
Support my alcoholism by getting some Star Stuff stuff MERCH:
/ @dylanodonnell
Dylan’s links
----
KZfaq: / dylanodonnell
PATREON : / dylanodonnell
WEB : byronbayobservatory.com.au
FACEBOOK : / dylanodonnell
TWITTER : / erfmufn
INSTAGRAM : / dylan_odonnell_
APPLE MUSIC : apple.co/3Ya2cHM
SPOTIFY : spoti.fi/3Ye8G8w
Intro Music - "Moving the Ocean (Aerologic Edit)" by Blastculture.
Outro Music - "Trap Epic" by Simon Jomphe Lepine via Storyblocks

Пікірлер: 108

@DylanODonnell Ай бұрын

SMALL CORRECTION - my stretched image is NON-LINEAR .. not linear like I say in the video and I shouldn't have left the deepSNR box ticked... thanks for calling me out on my error! The speckle however likely doesn't come from this but from DeepSNR being trained on drizzled noise scales which @Ekuy1 notes in the comments. Scaling down matched the noise scale so the model was better able to denoise smoothly - interesting!

@8gonzalo411 Ай бұрын

Are you going to photograph M58 to test deepSNR without the linear data option? 🤔

@FAstroHD Ай бұрын

Loving the more regular content mate, keep em coming

@DylanODonnell Ай бұрын

Thanks!

@anvikshiki Ай бұрын

Beautiful image, Dylan! The nebulae you capture in there with HaGB are stunning!

@DylanODonnell Ай бұрын

Thanks!

@Ekuy1 Ай бұрын

dsnr is trained on properly drizzled data, and doesn't work well on standard integrations. this is because proper drizzle has scale 1 noise, where each pixel's noise profile is independent of the surrounding pixels (its more complicated but go with this for now), while standard integrations use various interpolation methods which will cause larger scale noise profiles, such as 2 or 4 pixel noise. due to the nature of dsnr, these clusters in the noise will be mistaken for stars, which will cause the "speckling" you see here. by resampling data down by 2x which has improperly interpolated, 2-pixel noise profiles, like you've done here, you essentially reduce the noise scale to single pixels, which is what dsnr has been trained on, and recognises as well pre-processed data. if you want your images at full resolution, and not scaled down, use WBPP drizzle, or another algorithm which *correctly* implements drizzle integration (Siril and DSS drizzle do not work, they still use surrounding-dependent interpolation), then dsnr will work perfectly well on full-resolution images, just like the results you're currently getting right now. also, drizzle definitely DOES work on CFA (or debayered) images, but likewise, it needs to be correctly drizzled. p.s. lots of misinformation and misconceptions around the drizzle and deepsnr algorithms on cloudynights, if you want to learn more, read the source documentation from the authors "Variable-Pixel Linear Reconstruction"

@DylanODonnell Ай бұрын

Ok that's very elucidating thanks! Mine wasn't a perfect scale because the drizzled image is 12,560 which I cropped in slightly first.. *then* resized to 4096 (so god knows what the division is) and then got the speckle.. then resampled from that to 2048 where it worked. But it sounds like I got close enough and what you're saying makes sense. That means it wasn't the "linear" checkbox after all - thank you!

@Ekuy1 Ай бұрын

@@DylanODonnell yeah, i would definitely recommend using drizzle during the pre-processing (stacking) workflow. WBPP is the best that ive used, and drizzle also has many other benefits as it was developed for hubble (ill list them below if youre interested). however drizzle integration is process intensive, and time consuming, ive had WBPP crash many times while running drizzle, due to insufficient memory (only have 32GB). as for your current data, you can try software binning it by 2x, or 4x exactly, because then it doesn't have to undergo any of the weird sampling stuff which happens when you input a custom resolution. make sure to select bilinear interpolation mode, so you are literally just taking 4 pixels and averaging them into 1. this way maybe you can save some resolution, but still get a good result with your current stack, but the best method is still to drizzle. another thing you could try is to actually add noise. it sounds weird but it can work. if you add gaussian noise using pixinsight's "NoiseGenerator" process, deepsnr can actually work well on that, it was the original method of using dsnr on mono data before someone thought up creating 3 independent stacks, but it can save colour data too! if you're interested here are some of the reasons why drizzle was developed for the HST - It preserves photometry and resolution of the input image (remember the scales of noise i talked about earlier? this clumping affects your whole image, so your resolution is compromised in standard integrations!) - drizzle weighs input image pixels based on their statistical significance, and so removes effects of geometric distortion on photometry. however im not sure if this is properly implemented into amateur preprocessing software. im sure theres people who know more than me that could tell you

@Ekuy1 Ай бұрын

@@DylanODonnell dont mind me yapping for ages 😭

@DylanODonnell Ай бұрын

I do drizzle .. not sure if I wasn’t clear on that .. I always drizzle .. I have a video about that too :)

@Ekuy1 Ай бұрын

@@DylanODonnell what algorithm do you use. wbpp?

@newpilot4370 Ай бұрын

Great vid, as always! Looks like another good plug-in for my PixInsight. Thanks, buddy.

@Hot_Sky_Astronomy Ай бұрын

Dylan, with pixinsight's WBPP stacking menu, use these settings- Drizzle scale 1x Drop Shrink 0.85 Varshape = 1.5 Grid Size is variable based on dither distance. I dither 30px at 0.58"/Px, so I use Grid Size of 8. if you're under that, consider 16, or even 24 or 32. Increased Dither with Drizzle and DeepSNR will give you a massive boost in SNR and detail, I've measured an increase of ~5-10% in both sharpness and SNR, plus you get the ultra clean drizzle background.

@DylanODonnell Ай бұрын

Thanks for the tip !

@EarthAndSky4u Ай бұрын

Thank you for teaching me over the last 4 yrs I have been loving this astro stuff! There are a couple others as well but wanted to say Thanks!

@DylanODonnell Ай бұрын

Hey mate thanks for saying so !

@johnmcbryde715 Ай бұрын

Goin' deep again! woohoo!

@_Astrovert Ай бұрын

I just tried this on a non-linear image and it worked beautifully! I had Starnet set up for GPU acceleration and this used the GPU as well with just a normal installation. Great software!

@DylanODonnell Ай бұрын

It’s great huh! You’re the second person who’s said GPU acceleration worked .. neat!

@MrGuilletv Ай бұрын

Thx for the educational video Dylan, from Argentina, another southern Chad sky

@constellationshots3893 Ай бұрын

DeepSNR is a great tool. I’m just eagerly waiting for gray scale support because I don’t want to have to run 3 separate integrations and run it on an rgb combination of that.

@daviddayag Ай бұрын

Awesome info man.

@DylanODonnell Ай бұрын

Cheers man !

@ronm6585 Ай бұрын

Thanks Dylan.

@kvzastro3753 Ай бұрын

if you CFA drizzle your data at 1x then the dots disappear when using DeepSNR so you can CFA drizzle your 10000 pixel-wide image at 1x and keep it at 10000 pixels wide and still get the super clean result :)

@MarcelBlattner Ай бұрын

Down sampling indeed makes everything better. 🎉

@user-lt9py2pu6u Ай бұрын

I never liked Topaz noise reduction. The biggest problem to be fair was probably my own inexperience and the fact I was running it on an older computer and OS which although according to Topaz was compatible, was not really up to it. When I upgraded to a new and much more powerful laptop with Windows 11 I started using Noise Exterminator which I find gives much better results. I bought this as a package with the star and gradient exterminator plugins which also work very well, I would have liked to have bought blur exterminator as well but as I don't use Pixinsight it wouldn't work. Nice image of M83, DeepSNR does look to give great results.

@mdnt_astro Ай бұрын

This is cool, I definitely need to try DeepSNR and downscaling earlier in the process. Disclaimer: I have *NO* idea what I'm doing most of the time lol. I usually start out with a bit of BlurX followed my NoiseX as the first two steps in processing. I'm nowhere near the level of astrophotographer as you are, but it seems like a good thing to do before using StarX or doing any stretching. Lately my processing has involved a lot of experimentation with the order of things I do to see what works best but I can't really land on a solid permanent order. After some basic stretching on both stars and background independently, I'll take things into photoshop where I'm a bit more familiar to finalize as separate layers. One of my last steps is usually hitting the background layer with another bit of NoiseX or Topaz and I'll play around with them both to see what I like better. I don't normally downscale until I'm exporting a final I’mage because I like the option of having a full resolution photo in case I need it later while being able to export smaller sizes for social media at the same time. Sometimes I wish there was an easy was to process every image the same way, but I guess if it was easy then everyone would do it.

@DylanODonnell Ай бұрын

Thanks! And yes my workflow will change from image to image a little depending on what it needs. I definitely find these steps work best at the end after getting to the scale you want for screen .. I don’t know why exactly but it’s worth experimenting in your workflow to test.

@MNoel-mu8mf Ай бұрын

Great video, instructive and clear! I was wondering if this software can also be installed in Siril...?

@DylanODonnell Ай бұрын

The author is working on updates so maybe !

@simonpepper5053 Ай бұрын

Thanks Dylan will check this out. Can you do a video on resampling and sizes as you mention here I am not doing this with drizzled data and you are making me think I should be? Thanks

@anandarochisha Ай бұрын

Wow !! That software is so good !! Why do we even need telescopes ANYMORE !! Just take a picture through your binoculars with your IPhone and PRESS THE HUBBLE BUTTON in our AI software !! LOVING IT !! 😅

@Miguel_Noppe Ай бұрын

Haven't used it yet, but I do use BlurX. But I'm very careful with that tool, especially with sharpening the stars, BlurX is usually too harsh so I set that very very low like 0,03-5.

@michaeledmonds3027 Ай бұрын

Another great video! Question, please: What de-noising software would you recommend for color cameras (ASI585MC PRO) using Siril? Thanks, Michael

@DylanODonnell Ай бұрын

Lotta people using graxpert .. my results have been mixed but I may be using it wrong. However deepsnr is fairly new so expect OSC support eventually :)

@michaeledmonds3027 Ай бұрын

Thanks for your near instant reply!@@DylanODonnell

@brianhayward8240 Ай бұрын

Based on y our video, I just downloaded DeepSNR and tried it out. I had previously configured Startnet2 and BXT to use GPU acceleration on Linux (and provided a simplified process online) I verified using the nvidia-settings tool in Linux that it was spiking my GPU to 75% utilization. So I assume if you have tensorflow already GPU accelerated, you automatically get GPU acceeleration for DeepSNR as well.

@DylanODonnell Ай бұрын

Ahh I imagined that could be the case.. that's great! It's quite slow without it. I could never get the Star Net GPU acceleration to work!

@Wombatzone31 Ай бұрын

I'm using a 294MC sensor with a Bintel 200mm Reflector. On NGC 6744 at 8288px both struggle to make any difference of value in the process, buttttt.... if I resample to just 50% both programs work a treat..... blur still taking the win tho for detail retained with one shot colour I find.

@DylanODonnell Ай бұрын

Good to hear you’ve noticed the same thing !

@Wombatzone31 Ай бұрын

@@DylanODonnell 75% gives a natural background without further processing I just tested.

@WilliamBlakers Ай бұрын

Wah😢. We lost space Gandalf today.

@DylanODonnell Ай бұрын

I just read :( he was a wonderful man and an amazing presenter at star stuff. I’ll miss him.

@michaeledmonds3027 Ай бұрын

Another question please sir: Is "Astro Pixel Processor" still a good choice as an integrated astro imaging software package? Or, is it a bit dated now with less flexibility? I'm using Siril right now and find that it works, but would like better use of third-party plug-ins. Thanks again, Michael

@CraigStocksArts Ай бұрын

Is there a reason you didn’t compare it to NoiseXTerminator? That seems such a logical point of comparison. (FWIW I use BXT on my images while linear and NXT and SXT at the end on nonlinear images; I never drizzle or down-sample.)

@ryanmichaelhaley Ай бұрын

Hey Dylan, I like how you added your butt crack for like half a second in the intro scene, I laughed. I have pretty much abandoned Topaz for blur exterminator. Is this new process better than blur exterminator?

@DylanODonnell Ай бұрын

I have been using both in conservative amounts at the end because they both have qualities I like (topaz for denoise and blurx for sharpening. I’d typically sharpen first because that introduces noise then denoise last to remove it.

@ryanmichaelhaley Ай бұрын

@@DylanODonnell That's interesting. I typically run Noise Xterminator first, then do a couple of other things, then run Blur Xterminator. I'll try it the other way around, makes sense to me.

@Fractalite Ай бұрын

Lovely image . Isn't it around 12 hours total integration ?

@DylanODonnell Ай бұрын

You’re absolutely right. King of the fact checkers! I can’t update the video but I’ve updated my socials and blog etc. thank you!

@Hubaround1 Ай бұрын

I had to re-watch with CC on to make sure I was hearing it right. You called it linear data (and the image title says linear) but you also said you had stretched it and used HT. In the next couple of sentences you called it non-linear which it seems to be since if it was linear and you were using STF the green bar would be showing. But when you ran the tool you kept the linear box checked which may or may not have impacted the results. A little confusing.

@DSOImager Ай бұрын

I caught that too. Maybe linear is non-linear down under? Bad joke.. lol.

@DylanODonnell Ай бұрын

You are absolutely correct and I am at fault here for mixing them up .. I've pinned a correction :)

@desbarry8414 Ай бұрын

Not for osc then? Im using Grad Xpert denoise currently. Is it as good as Noise Xterminator?

@DylanODonnell Ай бұрын

I haven’t compared 1:1 but it sounds like the author is working on OSC support

@stantonstebbins9305 Ай бұрын

@@DylanODonnellit works great for OSC if you make sure to specify in WBPP you are using CFA images in the calibration tab and do a 1x drizzle with a drop shrink of 0.6 to 0.9 (it will automatically do a CFA drizzle)

@SimonT65 Ай бұрын

Installing this broke my cuda acceleration, now my RC Astro tools have reverted back to running via the CPU

@DylanODonnell Ай бұрын

ouch!

@SimonT65 Ай бұрын

@@DylanODonnell Probably a good idea to put up a disclaimer telling people it may break your CUDA acceleration

@DylanODonnell Ай бұрын

@@SimonT65 There is someone else in the comments where it worked and DeepSNR is GPU accelerated!

@SimonT65 Ай бұрын

@@DylanODonnell I was using Russell Crowman's experimental PixInsight repository which bypassed having to install all the cuda toolkit etc. Maybe thats why it broke my cuda acceleration. Anyways, i've just spent 30 mins downloading and installing the full version as I couldn't get Russel's version to work again

@8gonzalo411 Ай бұрын

I was very happy photographing M83 with my 6" Newtonian and you ruined my night. Thanks but I didn't need to see that impressive image right now xD 🙈😅 I'm going to try it with Ha because I was doing RGB. Thanks for the idea Greetings from Argentina

@philleng480 Ай бұрын

How do you work for an Australian astro vendor and promote an American vendor? I guess you are private IT and you have done some work for Bintel?

@DylanODonnell Ай бұрын

Yeh I don’t work for Bintel. I own a digital agency. Also my audience are like 90% American !

@ws5246 Ай бұрын

Graxpert now has noise reduction it it.. give it a look

@icyxxxxx Ай бұрын

I saw u say it’s good on discord lol

@DylanODonnell Ай бұрын

Thanks /r/astrophotography :)

@icyxxxxx Ай бұрын

@@DylanODonnell yeaahh

@griffith500tvr Ай бұрын

Can you please try attaching one of your astro cameras to a kids telescope and take some astro photos, tracked of course, maybe you can get your hands on a shitty Tasco telescope from years ago.

@user-lt9py2pu6u Ай бұрын

Speaking of Tasco telescopes, unbelievably last year a retro shop down the street from my local pub that specializes in 1970s' stuff here in the UK had a 60mm alt-az mounted one on sale for, wait for it, £400. I fell over laughing until I walked passed a few days later to see it had been sold. I don't know who parted with good money for that piece of junk but I hope it was not someone who had just become interested in astronomy and was buying their first scope.

@griffith500tvr Ай бұрын

@@user-lt9py2pu6u I had two or three Tasco's, two 60mm refractors and a 4inch reflector, after that experience I did not go anywhere near another telescope for 15 years. Telescopes like that destroy the hobby.

@leonim513 Ай бұрын

DeepSNR is good, but why are you comparing it to BlurX, a deconvolution tool, and not NoiseX?

@DylanODonnell Ай бұрын

I compared it to topaz? I was talking generally about AI based tools generally when I added blurX at the end as a matter of workflow.

@leonim513 Ай бұрын

@@DylanODonnell I rewatched that part - fair, but I'm still not sure why you'd run BlurX on a nonlinear image.

@DylanODonnell Ай бұрын

@@leonim513I typically try both before and after stretch and resize, and it always works best after downsampling and stretching .. which is what I was saying. It may be the huge 12,000px size of my drizzled image that blurx, and starnet and deepsnr all do poorly with. That’s why I mentioned them all together.

@leonim513 Ай бұрын

@@DylanODonnell That's fair, and since tone doesn't come across in text well, I was asking from pure curiosity - not trying to criticize.

@DylanODonnell Ай бұрын

@@leonim513yeh man all good :)

@lawrencesaville3345 Ай бұрын

all was going swimmingly until you mentioned it doesn't work for OSC ...........................!

@DylanODonnell Ай бұрын

Coming soon hopefully :)

@stantonstebbins9305 Ай бұрын

In WBPP make sure you verify it recognized CFA images, then do 1x drizzle with drop shrink 0.6 to 0.9. Need to have enough dithering.

@steevebody449 Ай бұрын

I’ve been using this tool almost since it came out, and there are tricks to get it working well every time. First you must dither, then your drop shrink size is paramount for this to work well. This depends a lot on your pixel scale and camera. In the DeepSkyCollective (I process a lot of images for the group) we use drop shrink of 0.35 and varshap 1.5. I found this works very well for my image scale (1.05) when I drizzle 2x. If I use the regular 0.9 square drizzle I get these speckles like you. But only at drizzle 2x at drizzle 1x they are gone… so it requires a little trial and error. There is also a trick to use this on single mono images that are not combined. Essentially you need to create 3 separate stack for that filter, combine in RGB, denoise then convert to greyscale. You shouldn’t have to reduce the size like you are doing to denoise using this… you really want to apply this early, just after deconvolution.. you should check Charles tutorial on this www.nightphotons.com/guides/single-channel-denoise you should also check the DSC suite which helps with creating triple stacks easily pixinsight.com/forum/index.php?threads/🌍-announcing-the-dsc-processing-suite-empowering-collaborative-projects-🌎.22833/

@DylanODonnell Ай бұрын

@steevebody449 great tips .. thank you!

@TexasEngineerScotty Ай бұрын

Comment for DoD

@ghillan Ай бұрын

No osc cameras, no independent interface ( you HAVE to have PI ), no gpu acceleration. For me that uses OSC camera, Siril and Affinity Photo sounds great! 😭

@DylanODonnell Ай бұрын

It's early days :)

@vexari_ Ай бұрын

the next update of siril will support deepsnr along with spcc iirc

@DylanODonnell Ай бұрын

@@vexari_ wow there you go !

@dravmtp385 Ай бұрын

have you tried graxpert?, it does background removal and denoise. I've used it with OSC data and other processing with siril and gimp. Also had the same issues with the spotty pixels when you run it on large image sizes so i'm not going to claim it's perfect. It is free though and runs independently so you don't need PI.

@s.m8766 Ай бұрын

ACTIVATE WINDOWS

@DylanODonnell Ай бұрын

YOU’RE NOT MY DAD

@ozicryptoG Ай бұрын

I have it. I don't use it. Just like I don't use any AI or similar software .. not keen. But hey, Dylan. You do you on this one. I still very much respect your opinion. mid.

@DylanODonnell Ай бұрын

Cheers mate. 🍻