mlpack IRC logs, 2018-04-04

Logs for the day 2018-04-04 (starts at 0:00 UTC) are shown below.

April 2018
--- Log opened Wed Apr 04 00:00:21 2018
01:16 -!- vivekp [~vivek@unaffiliated/vivekp] has quit [Ping timeout: 276 seconds]
01:18 -!- vivekp [~vivek@unaffiliated/vivekp] has joined #mlpack
03:27 -!- sumedhghaisas [] has quit [Read error: Connection reset by peer]
03:29 -!- sumedhghaisas [] has joined #mlpack
03:38 -!- govg [~govg@unaffiliated/govg] has quit [Ping timeout: 240 seconds]
04:20 -!- manthan [6725c94b@gateway/web/freenode/ip.] has joined #mlpack
04:46 -!- govg [~govg@unaffiliated/govg] has joined #mlpack
05:07 -!- witness [uid10044@gateway/web/] has joined #mlpack
06:51 -!- sulan_ [] has joined #mlpack
06:56 -!- wenhao [731bc015@gateway/web/freenode/ip.] has joined #mlpack
07:09 -!- ImQ009 [~ImQ009@unaffiliated/imq009] has joined #mlpack
07:17 -!- witness [uid10044@gateway/web/] has quit [Quit: Connection closed for inactivity]
07:35 -!- sulan_ [] has quit [Quit: Leaving]
07:57 -!- rajeshdm9 [0e8b9b18@gateway/web/freenode/ip.] has joined #mlpack
08:01 -!- rajeshdm9 [0e8b9b18@gateway/web/freenode/ip.] has quit [Client Quit]
08:48 -!- sumedhghaisas [] has quit [Read error: Connection reset by peer]
08:50 -!- sumedhghaisas [~yaaic@] has joined #mlpack
09:03 -!- sumedhghaisas [~yaaic@] has quit [Read error: Connection reset by peer]
09:03 -!- sumedhghaisas [~yaaic@2a00:79e0:d:fd00:2012:35d1:8f21:b46a] has joined #mlpack
09:05 -!- sumedhghaisas2 [~yaaic@2a00:79e0:d:fd00:2012:35d1:8f21:b46a] has joined #mlpack
09:08 -!- sumedhghaisas [~yaaic@2a00:79e0:d:fd00:2012:35d1:8f21:b46a] has quit [Ping timeout: 276 seconds]
09:15 -!- Atharva [uid288001@gateway/web/] has joined #mlpack
10:00 -!- csoni [~csoni@] has joined #mlpack
10:07 -!- zoq_ [] has joined #mlpack
10:15 -!- vpal [~vivek@unaffiliated/vivekp] has joined #mlpack
10:16 -!- vpal is now known as vivekp
10:16 -!- Netsplit *.net <-> *.split quits: zoq
11:01 -!- csoni [~csoni@] has quit [Read error: Connection reset by peer]
11:01 -!- zoq_ is now known as zoq
11:09 -!- dmatt [~quassel@2001:648:2800:240:fd63:dfcc:ecee:1413] has joined #mlpack
11:25 -!- Atharva [uid288001@gateway/web/] has quit [Quit: Connection closed for inactivity]
11:29 -!- dmatt [~quassel@2001:648:2800:240:fd63:dfcc:ecee:1413] has quit [Remote host closed the connection]
13:19 -!- sumedhghaisas2 [~yaaic@2a00:79e0:d:fd00:2012:35d1:8f21:b46a] has quit [Read error: Connection reset by peer]
13:20 -!- sumedhghaisas [~yaaic@2a00:79e0:d:fd00:2012:35d1:8f21:b46a] has joined #mlpack
13:54 -!- govg [~govg@unaffiliated/govg] has quit [Quit: leaving]
14:31 -!- Atharva [uid288001@gateway/web/] has joined #mlpack
14:32 < Atharva> rcurtin: zoq: what irc client do you use to stay connected forever? Do you use it on phone or pc?
14:35 < rcurtin> Atharva: I use irssi in a GNU screen session on a server that I host (it's the same server that hosts
14:35 < rcurtin> when I read messages, I simply connect from whatever computer I am using with ssh and resume the screen session
14:36 < rcurtin> I think it is an unusual setup but it works for me :)
14:41 < Atharva> Oh, okay, I don’t have a server for that kind of a setup. I think I would have to just search the net for a good client.
14:42 < rcurtin> I know there are IRC bouncers out there... I want to say one of these is called 'matrix'? but I am not certain
14:53 < Atharva> Yeah, I have heard about the bouncers, not quite sure what they do. I will check them out. There are cloud based clients which keep you online all the time but they charge on a per month basis and are quite expensive.
14:53 < rcurtin> yeah, I thought there was at least one that was free
14:54 < rcurtin> but even if you don't have a way to have a client always in the room, it is logged to, so you can always keep an eye on that :)
14:56 < Atharva> That is extremely useful, I always use the logs.
14:57 < Atharva> I had this another doubt, how do I build everything else except the test framework. I am trying some changes in the mlpack ann codebase and some tests fail to compile because of that.
14:57 < rcurtin> you can configure cmake with -DBUILD_TESTS=OFF
14:58 < rcurtin> and then when you type 'make', by default, the tests will not be built (you should still be able to type 'make mlpack_test' if you want the tests)
15:01 < Atharva> It’s still failing, I think I will have to check the changes I made. How do I compile just the ANN module?
15:01 < rcurtin> well, so this one is a little bit tricky
15:01 < rcurtin> the ANN code doesn't actually compile into anything because it is header-only
15:01 < rcurtin> this is the case with a lot of code in mlpack (but not all of it---any .cpp files in src/mlpack/core and src/mlpack/methods get compiled into
15:02 < rcurtin> so the only way it gets compiled into something is either in the tests in src/mlpack/tests/ or in the bindings found in src/mlpack/methods/*/*_main.cpp
15:06 -!- sumedhghaisas [~yaaic@2a00:79e0:d:fd00:2012:35d1:8f21:b46a] has quit [Ping timeout: 276 seconds]
15:18 < rcurtin> ok... I think that I have mlpack pypi packages compiling successfully. once I verify they are working right I'll upload the scripts into the jenkins-conf repository
16:06 -!- wenhao [731bc015@gateway/web/freenode/ip.] has quit [Ping timeout: 260 seconds]
16:16 -!- vivekp [~vivek@unaffiliated/vivekp] has quit [Read error: Connection reset by peer]
16:21 -!- vivekp [~vivek@unaffiliated/vivekp] has joined #mlpack
16:29 -!- dmatt [~quassel@2001:648:2800:240:84b1:fd20:57e2:ec69] has joined #mlpack
17:18 -!- s1998_ [0e8bc409@gateway/web/freenode/ip.] has joined #mlpack
17:24 < s1998_> zoq: rcurtin: w.r.t PR 9 of models, I have changed the dataset to mnist (currently in csv but the train has size of 104MB, limit : 100MB). Should I read the data from original MNIST dataset (which was in bytes format) ?
17:25 < s1998_> Or should I break the train data (in csv) into two parts and then push the changes ?
17:27 < s1998_> Another thing is the current implementation reaches test accuracy of 82% but I think this can be fixed using (batch) normalization (since currently only l2 normalization is used). Should I do this (as in write code to find mean and sigma) or use batch norm layer ?
17:33 -!- dmatt_ [~quassel@2001:648:2800:240:413:55cb:13d4:4d2d] has joined #mlpack
17:36 -!- dmatt [~quassel@2001:648:2800:240:84b1:fd20:57e2:ec69] has quit [Ping timeout: 255 seconds]
17:36 -!- s1998_ [0e8bc409@gateway/web/freenode/ip.] has quit [Ping timeout: 260 seconds]
18:04 -!- s1998_ [0e8bc409@gateway/web/freenode/ip.] has joined #mlpack
18:15 -!- dmatt_ [~quassel@2001:648:2800:240:413:55cb:13d4:4d2d] has quit [Remote host closed the connection]
18:21 -!- daivik [dfbc0b3d@gateway/web/cgi-irc/] has joined #mlpack
18:40 -!- Atharva [uid288001@gateway/web/] has quit [Quit: Connection closed for inactivity]
18:42 -!- daivik [dfbc0b3d@gateway/web/cgi-irc/] has quit [Quit: - A hand crafted IRC client]
18:43 -!- daivik [ab4fc5ec@gateway/web/cgi-irc/] has joined #mlpack
19:03 -!- daivik [ab4fc5ec@gateway/web/cgi-irc/] has quit [Quit: - A hand crafted IRC client]
19:22 < zoq> Atharva: I use almost the same setup irssi + tmux.
19:22 < zoq> s1998_: Testing the byte format sounds reasonable to me, hdf5 might be another solution.
19:23 < zoq> s1998_: About the accuracy, I think there are multiple options we could test, different architecture, l2 norm, batchnorm, another optimizer, so if you like please feel free to look into some options.
19:49 -!- s1998_ [0e8bc409@gateway/web/freenode/ip.] has quit [Ping timeout: 260 seconds]
19:57 -!- witness [uid10044@gateway/web/] has joined #mlpack
20:16 -!- ImQ009 [~ImQ009@unaffiliated/imq009] has quit [Quit: Leaving]
20:32 < manthan> rcurtin : when exactly is the gradient() called for a differentiable layer?
20:33 < zoq> manthan: After the backward step.
20:33 < manthan> i mean what exactly will be the difference in the backward and gradient for a layer?
20:33 < manthan> backward will contain the update rule for backward pass
20:33 < manthan> what will gradient contain exactly?
20:36 < zoq> The update step for the parameter, you could merge both steps into one, but in this case you would have to run the backward step (error calculation) for the first layer as well, which is unnecessary since the error isn't going to be used.
20:38 -!- daivik [ab4fc910@gateway/web/cgi-irc/] has joined #mlpack
20:38 < manthan> so the error obtained in this function is the error upto the present layer and we have to write the logic for updating the parameter given the error and input?
20:38 < zoq> correct
20:39 < rcurtin> zoq: would it be right to say that Backward() is the derivative of the inputs with respect to the error, whereas Gradient() is the derivative of the parameters with respect to the error?
20:39 < rcurtin> or to be clear, "Backward() is the derivative of the inputs of a particular layer with respect to the backpropagated error"
20:40 < zoq> yes, could could say that
20:40 < rcurtin> ok, just making sure---when I realized that it made the whole system a lot more clear to me, but I wasn't sure if I was correct :)
20:41 < zoq> Might be a good idea, to clarify that in the tutorial.
20:41 < zoq> Will set that on the list.
20:41 < manthan> shouldnt backward() be derivative of the backpropagated error with respect to the present layer paramters?
20:42 < manthan> so that i can backpropogate this error to previous layer
20:46 < manthan> for eg :- for ith layer, w(i)(new) = w(i)(previous) + alpha*dL/dw(i) and backpropogated error to previous layer will be dL/dw(i+1) * dw(i+1)/dwi ?
20:49 < manthan> so backward() implements logic for finding dw(i+1)/dwi given dL/dw(i+1)
20:49 < zoq> of a particular layer, in case of a ffn it's the previous one
20:49 < manthan> is this correct?^
20:51 < zoq> yes, looks correct to me
20:52 -!- daivik [ab4fc910@gateway/web/cgi-irc/] has quit [Quit: - A hand crafted IRC client]
20:52 < zoq> haven't checked the backward/gradient step of the flexible ReLU layer.
20:52 < zoq> Will take a closer look at the code in the next days.
20:52 -!- daivik [ab4fdd48@gateway/web/cgi-irc/] has joined #mlpack
20:53 < rcurtin> the backward step looked correct to me when I did the previous review, but I am not 100% certain, only about 95% :)
20:53 < manthan> i think backward is correct but i am not sure about gradient
20:53 < manthan> this is because i am not able to clearly understand what gradient should contain
20:54 < manthan> backward is clear to me now^
20:54 < rcurtin> I think the gradient here should contain just one element, d L / d alpha
20:55 < manthan> yes it contains one element but what is error in this case which the function obtains as a function argument?
20:55 < manthan> i mean gradient() function^
21:10 < manthan> backward() - derivative of backprop error with respect to the input and gradient() - derivative of error with respect to the trainable parameter and what i wrote for backward above should be true for gradient()
21:10 < manthan> this is what the definitions look like from the various trainable layers that i saw
21:13 < manthan> with this, gradient() of flexible relu layer should always be 1 as flexible relu is (max(0,x) +a)
21:22 -!- daivik [ab4fdd48@gateway/web/cgi-irc/] has quit [Quit: - A hand crafted IRC client]
21:32 < manthan> rcurtin : zoq : i have updated the flexible relu gradient function now, pls have a look. the concept is clear to me now. Thanks.
21:33 < manthan> i think addition of this in the tutorial will be very useful for contributors :D
22:06 -!- witness [uid10044@gateway/web/] has quit [Quit: Connection closed for inactivity]
22:14 -!- manthan [6725c94b@gateway/web/freenode/ip.] has quit [Ping timeout: 260 seconds]
--- Log closed Thu Apr 05 00:00:23 2018