Gauss-Newton local convergence
$begingroup$
Is it possible for the Gauss-Newton algorithm to converge to a local minima? How does convergence of Gauss-Newton to the correct minima compare with gradient decent and Levenberg-Marquardt?
numerical-methods nonlinear-optimization
asked Jan 22 at 19:15
John MJohn M
206
$endgroup$
add a comment |
$begingroup$
Is it possible for the Gauss-Newton algorithm to converge to a local minima? How does convergence of Gauss-Newton to the correct minima compare with gradient decent and Levenberg-Marquardt?
numerical-methods nonlinear-optimization
asked Jan 22 at 19:15
John MJohn M
206
$endgroup$
add a comment |
$begingroup$
Is it possible for the Gauss-Newton algorithm to converge to a local minima? How does convergence of Gauss-Newton to the correct minima compare with gradient decent and Levenberg-Marquardt?
numerical-methods nonlinear-optimization
asked Jan 22 at 19:15
John MJohn M
206
$endgroup$
Is it possible for the Gauss-Newton algorithm to converge to a local minima? How does convergence of Gauss-Newton to the correct minima compare with gradient decent and Levenberg-Marquardt?
numerical-methods nonlinear-optimization
numerical-methods nonlinear-optimization
asked Jan 22 at 19:15
John MJohn M
206
asked Jan 22 at 19:15
John MJohn M
206
asked Jan 22 at 19:15
John MJohn M
206
asked Jan 22 at 19:15
John MJohn M
206
asked Jan 22 at 19:15
John MJohn M
206
206
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
All three of those are "local" methods, which means that they will be attracted to local minima if they start close enough to them even if they are not global minima.
Non-convex optimization is just difficult in general. There is no general recipe to follow to guarantee getting the correct minimum in a reasonable amount of computation time. (Certain stochastic methods will always work eventually, but keeping runtime down is difficult.)
answered Jan 22 at 19:20
IanIan
68.5k25388
$endgroup$
$begingroup$
Are there existing methods that will converge to a course global minimum? After that, one could switch to one of the above approximations.
$endgroup$
– John M
Jan 22 at 19:44
$begingroup$
@JohnM The hard part is getting anywhere near the global minimum, sharply resolving it from inside a neighborhood is usually easy.
$endgroup$
– Ian
Jan 22 at 20:16
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3083581%2fgauss-newton-local-convergence%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
All three of those are "local" methods, which means that they will be attracted to local minima if they start close enough to them even if they are not global minima.
Non-convex optimization is just difficult in general. There is no general recipe to follow to guarantee getting the correct minimum in a reasonable amount of computation time. (Certain stochastic methods will always work eventually, but keeping runtime down is difficult.)
answered Jan 22 at 19:20
IanIan
68.5k25388
$endgroup$
$begingroup$
Are there existing methods that will converge to a course global minimum? After that, one could switch to one of the above approximations.
$endgroup$
– John M
Jan 22 at 19:44
$begingroup$
@JohnM The hard part is getting anywhere near the global minimum, sharply resolving it from inside a neighborhood is usually easy.
$endgroup$
– Ian
Jan 22 at 20:16
add a comment |
$begingroup$
All three of those are "local" methods, which means that they will be attracted to local minima if they start close enough to them even if they are not global minima.
Non-convex optimization is just difficult in general. There is no general recipe to follow to guarantee getting the correct minimum in a reasonable amount of computation time. (Certain stochastic methods will always work eventually, but keeping runtime down is difficult.)
answered Jan 22 at 19:20
IanIan
68.5k25388
$endgroup$
$begingroup$
Are there existing methods that will converge to a course global minimum? After that, one could switch to one of the above approximations.
$endgroup$
– John M
Jan 22 at 19:44
$begingroup$
@JohnM The hard part is getting anywhere near the global minimum, sharply resolving it from inside a neighborhood is usually easy.
$endgroup$
– Ian
Jan 22 at 20:16
add a comment |
$begingroup$
All three of those are "local" methods, which means that they will be attracted to local minima if they start close enough to them even if they are not global minima.
Non-convex optimization is just difficult in general. There is no general recipe to follow to guarantee getting the correct minimum in a reasonable amount of computation time. (Certain stochastic methods will always work eventually, but keeping runtime down is difficult.)
answered Jan 22 at 19:20
IanIan
68.5k25388
$endgroup$
All three of those are "local" methods, which means that they will be attracted to local minima if they start close enough to them even if they are not global minima.
Non-convex optimization is just difficult in general. There is no general recipe to follow to guarantee getting the correct minimum in a reasonable amount of computation time. (Certain stochastic methods will always work eventually, but keeping runtime down is difficult.)
answered Jan 22 at 19:20
IanIan
68.5k25388
answered Jan 22 at 19:20
IanIan
68.5k25388
answered Jan 22 at 19:20
IanIan
68.5k25388
answered Jan 22 at 19:20
IanIan
68.5k25388
68.5k25388
$begingroup$
Are there existing methods that will converge to a course global minimum? After that, one could switch to one of the above approximations.
$endgroup$
– John M
Jan 22 at 19:44
$begingroup$
@JohnM The hard part is getting anywhere near the global minimum, sharply resolving it from inside a neighborhood is usually easy.
$endgroup$
– Ian
Jan 22 at 20:16
add a comment |
$begingroup$
Are there existing methods that will converge to a course global minimum? After that, one could switch to one of the above approximations.
$endgroup$
– John M
Jan 22 at 19:44
$begingroup$
@JohnM The hard part is getting anywhere near the global minimum, sharply resolving it from inside a neighborhood is usually easy.
$endgroup$
– Ian
Jan 22 at 20:16
$begingroup$
Are there existing methods that will converge to a course global minimum? After that, one could switch to one of the above approximations.
$endgroup$
– John M
Jan 22 at 19:44
$begingroup$
Are there existing methods that will converge to a course global minimum? After that, one could switch to one of the above approximations.
$endgroup$
– John M
Jan 22 at 19:44
$begingroup$
@JohnM The hard part is getting anywhere near the global minimum, sharply resolving it from inside a neighborhood is usually easy.
$endgroup$
– Ian
Jan 22 at 20:16
$begingroup$
@JohnM The hard part is getting anywhere near the global minimum, sharply resolving it from inside a neighborhood is usually easy.
$endgroup$
– Ian
Jan 22 at 20:16
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3083581%2fgauss-newton-local-convergence%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
