mlpack IRC logs, 2018-07-25
Logs for the day 2018-07-25 (starts at 0:00 UTC) are shown below.
--- Log opened Wed Jul 25 00:00:01 2018
02:18 -!- Samir [~Samir__@18.104.22.168] has quit [Ping timeout: 256 seconds]
02:30 -!- Samir [~Samir__@22.214.171.124] has joined #mlpack
02:30 -!- Samir is now known as Guest29252
02:30 -!- Guest29252 is now known as Samir
05:32 < ShikharJ> zoq: I'll spend the rest of the week implementing the test for SSRBM, and push in the PR when the existing one is merged. I'll look for optimizations as well, though the code already looks pretty optimized to me.
05:38 < zoq> ShikharJ: Sounds like a good plan to me, will think optimizations too.
07:12 -!- vivekp [~vivek@unaffiliated/vivekp] has quit [Ping timeout: 276 seconds]
07:31 -!- vivekp [~vivek@unaffiliated/vivekp] has joined #mlpack
09:47 < Atharva> sumedhghaisas: you there?
09:49 -!- zoq_ [~firstname.lastname@example.org] has joined #mlpack
09:50 -!- zoq [~email@example.com] has quit [Read error: Connection reset by peer]
09:54 -!- vivekp [~vivek@unaffiliated/vivekp] has quit [Read error: Connection reset by peer]
09:55 -!- vivekp [~vivek@unaffiliated/vivekp] has joined #mlpack
09:56 < sumedhghaisas> Atharva: Hi Atharva
09:57 < Atharva> sumedhghaisas: I tested the reconstruction loss with conv network and it works!
09:57 < Atharva> on bernoulli
09:57 < sumedhghaisas> great!!!!
09:58 < sumedhghaisas> huh... phewww
09:58 < Atharva> But still, the error went down to 100 with 25 kl, so the samples weren't that good, but some were
09:58 < sumedhghaisas> 100?? it doesn't go lower?
09:58 < Atharva> I am currently training it on the feedforward model
09:58 < Atharva> to be sure
09:58 < Atharva> no
09:58 < sumedhghaisas> could you try for another architecture please? just to check
09:59 < Atharva> Should we get a better conv architecture
09:59 < Atharva> Yeah
09:59 < sumedhghaisas> I know the values for that arch
09:59 < Atharva> Can you suggest some architecture or point me to some link?
09:59 < sumedhghaisas> yeah... i thinking how do I send you that arch
10:01 < Atharva> maybe write it on a paper and send me a photo on hangouts
10:02 < sumedhghaisas> conv2d = snt.nets.ConvNet2D( output_channels=[32, 32, 32, 64], kernel_shapes=[[3, 3]], strides=[1, 2, 1, 2], paddings=[snt.SAME], activate_final=True, initializers=INITIALIZER_DICT, activation=tf.nn.relu, use_batch_norm=False) # convolved [7, 7, 64] convolved = conv2d(input_) # output is [8, 8, 64] output = tf.pad(convolved, [[0, 0], [0, 1], [0, 1]
10:02 < sumedhghaisas> this doesn't work :P
10:02 < sumedhghaisas> okay I will mail it to you
10:03 < sumedhghaisas> Atharva: Sent a mail
10:03 < sumedhghaisas> the arch contains sonnet definition but its easy to decode
10:03 < Atharva> okay, i will check
10:03 < sumedhghaisas> let me know if you don't understand anything in that definition
10:15 < Atharva> sumedhghaisas: Sorry, I can't understand it fully. Is it only the encoder you mailed me?
10:15 < sumedhghaisas> The decoder will be the exact reverse of this
10:17 < Atharva> Okay, what does this mean `output_channels=[32, 32, 32, 64]` Only 64 seems to be the number of output channels
10:17 < sumedhghaisas> umm... okay so the input is in shape [batch, height, width, channels]
10:18 < sumedhghaisas> now we keep applying a conv layer and a non linearity
10:18 < sumedhghaisas> so [32, 32, 32, 64] means 4 conv + ReLu layers
10:19 < sumedhghaisas> their output channels are respective
10:19 < Atharva> Okayy
10:19 < sumedhghaisas> all 4 layers will have common stride [3, 3]
10:20 < sumedhghaisas> and no padding will be applied
10:20 < Atharva> paddings=[snt.SAME]
10:20 < Atharva> this is mentioned, how will it remain SAME then?
10:20 < sumedhghaisas> yes... that means no padding
10:21 < sumedhghaisas> ahh wait... i was wrong
10:21 < sumedhghaisas> strides are [1, 2, 1, 2]
10:21 < sumedhghaisas> kernal shape is [3, 3]
10:22 < Atharva> I have a doubt, doesn't SAME padding mean that the output will have the same height and width as the input?
10:24 < sumedhghaisas> so I can write down the output shape [-1, 28, 28, 1] -> [-1, 28, 28, 32] -> [-1, 14, 14, 32] -> [-1, 14, 14, 32] -> [-1, 7, 7, 64]
10:25 < Atharva> Yes, so achieve 28, 28 -> 28, 28, we will need some padding right?
10:25 < sumedhghaisas> ahh yes... zero padding
10:26 < sumedhghaisas> I read VALID
10:26 < sumedhghaisas> I usually get confused between the 2
10:27 < Atharva> Haha I just remember it as SAME padding means the height and width will be same
10:27 < sumedhghaisas> Actually we should have this kind of API to define a ConvNet
10:27 < sumedhghaisas> this is usual in research
10:29 < Atharva> Yeah, maybe we should
10:29 < Atharva> This is a big network, it will take time to train. I think it will also be useful for celebA
10:30 < Atharva> I downloaded the dataset yesterday
10:31 < Atharva> Also, about conditional VAE, how should be append labels to the data when we use conv nets?
10:36 < sumedhghaisas> hmm... there are many ways for that, simplest is to treat it like another channels
10:36 < sumedhghaisas> are you starting with conditional VAEs as well?
10:39 < Atharva> I am planning to do it before i go to celebA
10:39 < sumedhghaisas> I would suggest the reverse
10:40 < sumedhghaisas> conditional VAE might be little harder and we will run out of time
10:41 < Atharva> I don't think they will be that hard, we just need to append labels, right?
10:41 < Atharva> I just need to change repar layer a bit
10:42 < sumedhghaisas> wouldn't be that hard, thats true
10:42 < sumedhghaisas> why would we need to change the repar?
10:43 < sumedhghaisas> we should also get the open PRs in shape. :)
10:43 < Atharva> We would need to add the labels to the output of the repar layer as well, wouldn't we?
10:44 < Atharva> Yeah, I will do that today, reviewing the PRs. Also, I will open another PR for the last commits I put in the reconstruction loss PR
10:44 < Atharva> Also, do you think we can merge that PR now?
10:45 < sumedhghaisas> I am not sure if we need to change anything in Repar layer but I have to think about that again
10:45 < sumedhghaisas> and yes, about the ReconstructLoss
10:45 < sumedhghaisas> we can merge that PR. :)
10:45 < sumedhghaisas> if we remove the extra commits
10:46 < sumedhghaisas> we need to also remove NormalDistribution from it
10:46 < Atharva> Yeah, I will do that in a while
10:46 < Atharva> Oh, why?
10:47 < sumedhghaisas> Its not exactly required and we still haven't figured out theproblem with that distribution in VAE
10:49 < sumedhghaisas> We shouldn't keep that distribution if we aren't sure it works with VAE
10:50 < Atharva> Okay
11:48 < Atharva> sumedhghaisas: I was just thinking, as it's a an ANN dists folder, we haven't specifically said that it's for VAEs, should we keep the normal distribution?
12:21 < sumedhghaisas> umm... we could. But where will it be used the current moment?
12:21 < sumedhghaisas> Atharva:
12:25 < Atharva> sumedhghaisas: It won't be used anywhere atleast for now
12:26 < Atharva> Another thing, the results aren't very good with bernoulli either
12:26 < Atharva> I am hoping they would get better as I train it more
12:26 < sumedhghaisas> But for bernoulli you are using BinaryMinst right?
12:27 < Atharva> Yes
12:27 < sumedhghaisas> Did you check if the binary mnist images look okay?
12:27 < Atharva> Yeah, they look fine
12:27 < sumedhghaisas> great. So what is the loss you are getting?
12:27 < Atharva> wait I will send it to you on hangouts
12:28 < sumedhghaisas> Sure thing
12:28 < Atharva> the loss now is 130 out of which 8 is kl, this is for feedforward nets
12:29 < sumedhghaisas> And does it seem to be going down?
12:29 < sumedhghaisas> although kl of 8 does not seem correct
12:29 < sumedhghaisas> should be higher
12:29 < Atharva> Yes, it's still going down
12:31 < Atharva> I have sent an image on hangouts
12:31 < sumedhghaisas> I know we have checked this... but could you check again if we are summing the correct dimension for KL?
12:31 < sumedhghaisas> the first dimension should be summed and 0th dimension should be meaned
12:31 < Atharva> sumedhghaisas: But that's how it has been for feedforward vae models with normal MNIST as well, and the results were good
12:32 < Atharva> the kl then too was under 10
12:32 < sumedhghaisas> I agree... Wait but we haven't changed MeanSquaredError right?
12:33 < sumedhghaisas> I am sensing some mistake here
12:33 < sumedhghaisas> so lets see
12:33 < Atharva> Sorry, what change?
12:33 < sumedhghaisas> so mean_squared_loss is (total_batch_loss) / (batch_size * num_attributes)
12:33 < sumedhghaisas> is that right?
12:33 < sumedhghaisas> I think so
12:34 < Atharva> but I changed it to (total_batch_loss) / (batch_size)
12:34 < Atharva> locally
12:35 < sumedhghaisas> okay... so KL_loss is (total_loss) / (batch_size)
12:36 < sumedhghaisas> this would be awkward for other users though
12:38 < sumedhghaisas> I think we should shift to (total_loss) / (batch_size) for MeanSquaredLoss as well
12:47 < Atharva> I changed Meansquared to (total_batch_loss) / (batch_size) as well
12:47 < Atharva> locally
14:08 -!- travis-ci [~firstname.lastname@example.org] has joined #mlpack
14:08 < travis-ci> manish7294/mlpack#15 (tree - 7116beb : Manish): The build is still failing.
14:08 < travis-ci> Change view : https://github.com/manish7294/mlpack/compare/12e9a36362b1...7116beb24b17
14:08 < travis-ci> Build details : https://travis-ci.org/manish7294/mlpack/builds/408053150
14:08 -!- travis-ci [~email@example.com] has left #mlpack 
14:08 -!- ImQ009 [~ImQ009@unaffiliated/imq009] has joined #mlpack
14:12 -!- travis-ci [~firstname.lastname@example.org] has joined #mlpack
14:12 < travis-ci> manish7294/mlpack#80 (tree - 7116beb : Manish): The build is still failing.
14:12 < travis-ci> Change view : https://github.com/manish7294/mlpack/compare/12e9a36362b1...7116beb24b17
14:12 < travis-ci> Build details : https://travis-ci.com/manish7294/mlpack/builds/79972745
14:12 -!- travis-ci [~email@example.com] has left #mlpack 
14:20 -!- zoq_ is now known as zoq
15:11 -!- chrisanthamum [6f5c4c10@gateway/web/freenode/ip.126.96.36.199] has joined #mlpack
15:40 -!- yaswagner [4283a544@gateway/web/freenode/ip.188.8.131.52] has joined #mlpack
16:09 -!- pd09041999 [~firstname.lastname@example.org] has joined #mlpack
17:16 -!- wenhao [db897713@gateway/web/freenode/ip.184.108.40.206] has joined #mlpack
17:26 -!- pd09041999 [~email@example.com] has quit [Ping timeout: 268 seconds]
19:15 -!- chrisanthamum [6f5c4c10@gateway/web/freenode/ip.220.127.116.11] has quit [Ping timeout: 252 seconds]
19:44 < rcurtin> ok, just an update from my end, I still don't have any clarity on the Symantec build systems yet... I simply haven't gotten a response
19:45 < rcurtin> also, there is interest in the new company in mlpack, so I think the first work I do there will be to write Julia wrappers for mlpack
19:45 < rcurtin> I saw that the Julia machine learning software offerings are quite small, so I think that mlpack can be a nice addition that will see adoption in that community
19:45 < ShikharJ> rcurtin: Nice idea.
19:46 < rcurtin> there are neural network toolkits available already for Julia (most wrapped from other languages), but lots of "traditional" techniques like HMMs, GMMs, nearest neighbor search, and others don't seem to be readily available
19:47 < rcurtin> ShikharJ: thanks, I hope that the company will not be the only users of the bindings :)
20:06 -!- ImQ009 [~ImQ009@unaffiliated/imq009] has quit [Quit: Leaving]
20:48 [Users #mlpack]
20:48 [ Atharva ] [ jenkins-mlpack ] [ petris ] [ Samir ] [ vivekp] [ xa0 ]
20:48 [ cjlcarvalho] [ jenkins-mlpack2] [ prakhar_code[m]] [ ShikharJ ] [ wenhao] [ yaswagner]
20:48 [ gtank_ ] [ lozhnikov ] [ rcurtin ] [ sumedhghaisas] [ wiking] [ zoq ]
20:48 -!- Irssi: #mlpack: Total of 18 nicks [0 ops, 0 halfops, 0 voices, 18 normal]
21:52 < zoq> rcurtin: I guess, in this case, no response is a good thing, what are the chances they forget they existed.
21:52 < zoq> rcurtin: I'm wondering, is julia a 'database' language?
21:53 < zoq> ShikharJ: Do you need any help with the SSRBM test?
22:07 -!- yaswagner [4283a544@gateway/web/freenode/ip.18.104.22.168] has quit [Quit: Page closed]
--- Log closed Thu Jul 26 00:00:02 2018