Kdtyjb

Space in array system equations

PTIJ: Why can't I eat anything?

Does splitting a potentially monolithic application into several smaller ones help prevent bugs?

Why does Deadpool say "You're welcome, Canada," after shooting Ryan Reynolds in the end credits?

The bar has been raised

Can someone explain what is being said here in color publishing in the American Mathematical Monthly?

"One can do his homework in the library"

What wound would be of little consequence to a biped but terrible for a quadruped?

Solving "Resistance between two nodes on a grid" problem in Mathematica

How much attack damage does the AC boost from a shield prevent on average?

Good allowance savings plan?

How much stiffer are 23c tires over 28c?

Built-In Shelves/Bookcases - IKEA vs Built

Make a transparent 448*448 image

Is Gradient Descent central to every optimizer?

Why the color red for the Republican Party

Replacing Windows 7 security updates with anti-virus?

Why would one plane in this picture not have gear down yet?

String reversal in Python

Finding algorithms of QGIS commands?

How do I deal with a powergamer in a game full of beginners in a school club?

If the Captain's screens are out, does he switch seats with the co-pilot?

Do items de-spawn in Diablo?

Rejected in 4th interview round citing insufficient years of experience

What are the best books to study Neural Networks from a purely mathematical perspective?

Modern Mathematical Theory for Neural Networks, Cellular Automata, NeuroscienceTwo-layer Perceptron for XORdelay time from mathematical perspectiveReference Request: Neural Networks as Mathematical ModelsSources to learn and understand advanced probability in ML modelsHow can logicians benefit from development of artificial neural networks?Deriving diagonal approximation of Hessian in a neural networkNeural Networks and the Chain RuleWhat are some current best practices for function approximation using neural networks?What does oplus symbol ⊕ do for 2 images in Convolution Neural Networks

5

3

$begingroup$

I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).

Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?

matrices book-recommendation mathematical-modeling neural-networks

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

5

3

$begingroup$

I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).

Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?

matrices book-recommendation mathematical-modeling neural-networks

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

$begingroup$

I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).

Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?

matrices book-recommendation mathematical-modeling neural-networks

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 2 hours ago

EliEli

341

2 Answers
2

active

oldest

votes

3

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

add a comment | 

2

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
  $endgroup$
  – Eli
  2 hours ago
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

Eli is a new contributor. Be nice, and check out our Code of Conduct.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3146016%2fwhat-are-the-best-books-to-study-neural-networks-from-a-purely-mathematical-pers%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

3

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

add a comment | 

3

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

add a comment | 

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

2

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
  $endgroup$
  – Eli
  2 hours ago
  

  
  
  
  

add a comment | 

2

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
  $endgroup$
  – Eli
  2 hours ago
  

  
  
  
  

add a comment | 

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144