Adding SubtractOperation and AverageOperation#25
Conversation
| class AverageOperation : public Operation<T> { | ||
| public: | ||
|
|
||
| vector<Tensor<T*>> tensors; |
There was a problem hiding this comment.
Thanks for the pull request!! This is a prelim review.
The representation of the Average Tensor will need to be changed I think. Generally how the network takes in a tensor is a single tensor with size [batch_size, input_dim]. SO the average operation should take as input t1 which would be a tensor and another int variable called axis specifying what dimension to average along.
Hence the API would look like
loss = losses::mse(y,gt)
averageLoss = tensorOps::average(loss,axis=0)
// and then continue from hereThis would be inline how pytorch, tf and numpy handle some operation along a list of objects. Hence, I think it would be better to adhere with their API.
There was a problem hiding this comment.
Right, I missed the big picture here. Thanks for pointing this out.
There was a problem hiding this comment.
No problem, looking forward to your updates :)
| #ifndef __OP_SUBTRACT_IMPL_INCLUDED__ | ||
| #define __OP_SUBTRACT_IMPL_INCLUDED__ | ||
|
|
||
| template <typename T> |
There was a problem hiding this comment.
Comments on top of the function. Can remove comments inside function
| // Average | ||
| template<typename T> | ||
| Tensor<T>* average(vector<Tensor<T>*>& tensors) { | ||
| if (tensors.size() == 0) { // To avoid division by zero. Should we do this? |
There was a problem hiding this comment.
Don't think this check will be needed if we adhere to the requested API
2 participants
The average operation will be useful in computing losses across a batch of data.