How to show negative entropy function $f(x)=xlog(x)$ is strongly convex?
Multi tool use
$begingroup$
Let $f: mathbb{R}_+ rightarrow mathbb{R}$ where $f(x)=xlog(x)$. How to show it is strongly convex, i.e.,
Definition: Let $f: mathbb{R}^n rightarrow mathbb{R}$ be differentiable. Then $f$ is strongly convex if $exists$ a positive constant $alpha > 0$ such that
$$
langle nabla f(y) - nabla f(x),y-x rangle geq alpha||y-x||^2 ,,,,,,,,,forall x,y in mathbb{R}^n tag{1}
$$
Following $(1)$ we have $(y-x)log(frac{y}{x})geq alpha(y-x)^2
,,,,,
forall x,y in mathbb{R}_+$.
What $alpha$ satisfies the above and how we can handle the inequality $forall x,y in mathbb{R}_+$ ?
inequality logarithms convex-analysis
asked Jan 17 at 17:52
SaeedSaeed
1,005310
$endgroup$
|
show 1 more comment
$begingroup$
Let $f: mathbb{R}_+ rightarrow mathbb{R}$ where $f(x)=xlog(x)$. How to show it is strongly convex, i.e.,
Definition: Let $f: mathbb{R}^n rightarrow mathbb{R}$ be differentiable. Then $f$ is strongly convex if $exists$ a positive constant $alpha > 0$ such that
$$
langle nabla f(y) - nabla f(x),y-x rangle geq alpha||y-x||^2 ,,,,,,,,,forall x,y in mathbb{R}^n tag{1}
$$
Following $(1)$ we have $(y-x)log(frac{y}{x})geq alpha(y-x)^2
,,,,,
forall x,y in mathbb{R}_+$.
What $alpha$ satisfies the above and how we can handle the inequality $forall x,y in mathbb{R}_+$ ?
inequality logarithms convex-analysis
asked Jan 17 at 17:52
SaeedSaeed
1,005310
$endgroup$
$begingroup$
Why don't you calculate the second derivative?
$endgroup$
– Dr. Sonnhard Graubner
Jan 17 at 17:53
$begingroup$
@Dr. Sonnhard Graubner : Because I want to better understand the definition to proof the general case:math.stackexchange.com/questions/3055187/… and solve it.
$endgroup$
– Saeed
Jan 17 at 17:57
$begingroup$
@Dr. Sonnhard Graubner : Also, the second derivative does not specify $alpha$. Actually, I do not know how to use this fact saying strongly convex implies $nabla^2f(x) succeq alpha I$ for this case to get $alpha$.
$endgroup$
– Saeed
Jan 17 at 18:01
$begingroup$
The case $x=1,,y=1+z>0$ gives $frac{ln(1+z)}{z}gealpha$, which for sufficiently large $z>0$ contradicts any proposed $alpha>0$.
$endgroup$
– J.G.
Jan 17 at 18:27
$begingroup$
@J.G. : I think I should have assumed that $0 leq x leq 1$?
$endgroup$
– Saeed
Jan 17 at 18:32
|
show 1 more comment
$begingroup$
Let $f: mathbb{R}_+ rightarrow mathbb{R}$ where $f(x)=xlog(x)$. How to show it is strongly convex, i.e.,
Definition: Let $f: mathbb{R}^n rightarrow mathbb{R}$ be differentiable. Then $f$ is strongly convex if $exists$ a positive constant $alpha > 0$ such that
$$
langle nabla f(y) - nabla f(x),y-x rangle geq alpha||y-x||^2 ,,,,,,,,,forall x,y in mathbb{R}^n tag{1}
$$
Following $(1)$ we have $(y-x)log(frac{y}{x})geq alpha(y-x)^2
,,,,,
forall x,y in mathbb{R}_+$.
What $alpha$ satisfies the above and how we can handle the inequality $forall x,y in mathbb{R}_+$ ?
inequality logarithms convex-analysis
asked Jan 17 at 17:52
SaeedSaeed
1,005310
$endgroup$
Let $f: mathbb{R}_+ rightarrow mathbb{R}$ where $f(x)=xlog(x)$. How to show it is strongly convex, i.e.,
Definition: Let $f: mathbb{R}^n rightarrow mathbb{R}$ be differentiable. Then $f$ is strongly convex if $exists$ a positive constant $alpha > 0$ such that
$$
langle nabla f(y) - nabla f(x),y-x rangle geq alpha||y-x||^2 ,,,,,,,,,forall x,y in mathbb{R}^n tag{1}
$$
Following $(1)$ we have $(y-x)log(frac{y}{x})geq alpha(y-x)^2
,,,,,
forall x,y in mathbb{R}_+$.
What $alpha$ satisfies the above and how we can handle the inequality $forall x,y in mathbb{R}_+$ ?
inequality logarithms convex-analysis
inequality logarithms convex-analysis
asked Jan 17 at 17:52
SaeedSaeed
1,005310
asked Jan 17 at 17:52
SaeedSaeed
1,005310
asked Jan 17 at 17:52
SaeedSaeed
1,005310
asked Jan 17 at 17:52
SaeedSaeed
1,005310
asked Jan 17 at 17:52
SaeedSaeed
1,005310
1,005310
$begingroup$
Why don't you calculate the second derivative?
$endgroup$
– Dr. Sonnhard Graubner
Jan 17 at 17:53
$begingroup$
@Dr. Sonnhard Graubner : Because I want to better understand the definition to proof the general case:math.stackexchange.com/questions/3055187/… and solve it.
$endgroup$
– Saeed
Jan 17 at 17:57
$begingroup$
@Dr. Sonnhard Graubner : Also, the second derivative does not specify $alpha$. Actually, I do not know how to use this fact saying strongly convex implies $nabla^2f(x) succeq alpha I$ for this case to get $alpha$.
$endgroup$
– Saeed
Jan 17 at 18:01
$begingroup$
The case $x=1,,y=1+z>0$ gives $frac{ln(1+z)}{z}gealpha$, which for sufficiently large $z>0$ contradicts any proposed $alpha>0$.
$endgroup$
– J.G.
Jan 17 at 18:27
$begingroup$
@J.G. : I think I should have assumed that $0 leq x leq 1$?
$endgroup$
– Saeed
Jan 17 at 18:32
|
show 1 more comment
$begingroup$
Why don't you calculate the second derivative?
$endgroup$
– Dr. Sonnhard Graubner
Jan 17 at 17:53
$begingroup$
@Dr. Sonnhard Graubner : Because I want to better understand the definition to proof the general case:math.stackexchange.com/questions/3055187/… and solve it.
$endgroup$
– Saeed
Jan 17 at 17:57
$begingroup$
@Dr. Sonnhard Graubner : Also, the second derivative does not specify $alpha$. Actually, I do not know how to use this fact saying strongly convex implies $nabla^2f(x) succeq alpha I$ for this case to get $alpha$.
$endgroup$
– Saeed
Jan 17 at 18:01
$begingroup$
The case $x=1,,y=1+z>0$ gives $frac{ln(1+z)}{z}gealpha$, which for sufficiently large $z>0$ contradicts any proposed $alpha>0$.
$endgroup$
– J.G.
Jan 17 at 18:27
$begingroup$
@J.G. : I think I should have assumed that $0 leq x leq 1$?
$endgroup$
– Saeed
Jan 17 at 18:32
$begingroup$
Why don't you calculate the second derivative?
$endgroup$
– Dr. Sonnhard Graubner
Jan 17 at 17:53
$begingroup$
Why don't you calculate the second derivative?
$endgroup$
– Dr. Sonnhard Graubner
Jan 17 at 17:53
$begingroup$
@Dr. Sonnhard Graubner : Because I want to better understand the definition to proof the general case:math.stackexchange.com/questions/3055187/… and solve it.
$endgroup$
– Saeed
Jan 17 at 17:57
$begingroup$
@Dr. Sonnhard Graubner : Because I want to better understand the definition to proof the general case:math.stackexchange.com/questions/3055187/… and solve it.
$endgroup$
– Saeed
Jan 17 at 17:57
$begingroup$
@Dr. Sonnhard Graubner : Also, the second derivative does not specify $alpha$. Actually, I do not know how to use this fact saying strongly convex implies $nabla^2f(x) succeq alpha I$ for this case to get $alpha$.
$endgroup$
– Saeed
Jan 17 at 18:01
$begingroup$
@Dr. Sonnhard Graubner : Also, the second derivative does not specify $alpha$. Actually, I do not know how to use this fact saying strongly convex implies $nabla^2f(x) succeq alpha I$ for this case to get $alpha$.
$endgroup$
– Saeed
Jan 17 at 18:01
$begingroup$
The case $x=1,,y=1+z>0$ gives $frac{ln(1+z)}{z}gealpha$, which for sufficiently large $z>0$ contradicts any proposed $alpha>0$.
$endgroup$
– J.G.
Jan 17 at 18:27
$begingroup$
The case $x=1,,y=1+z>0$ gives $frac{ln(1+z)}{z}gealpha$, which for sufficiently large $z>0$ contradicts any proposed $alpha>0$.
$endgroup$
– J.G.
Jan 17 at 18:27
$begingroup$
@J.G. : I think I should have assumed that $0 leq x leq 1$?
$endgroup$
– Saeed
Jan 17 at 18:32
$begingroup$
@J.G. : I think I should have assumed that $0 leq x leq 1$?
$endgroup$
– Saeed
Jan 17 at 18:32
|
show 1 more comment
1 Answer
1
active
oldest
votes
$begingroup$
We have $f'(x)=1+log(x)$. Strict convexity therefore means that there exists a strictly positive $alpha$ such that
$$[log(y)-log(x)](y-x)geqalpha(y-x)^2$$
holds. Without loss of generality I assume $y>x$. Then the above inequality requires that
$$log(y)-log(x)geqalpha(y-x).$$
Although you don't state it, I assume that the variables $x$ and $y$ live on $(0,1)$ (because they are probabilities). Since the slope of the $log$ function on the interval $(0,1)$ is larger than or equal to 1, you can choose any positive $alpha$ smaller than 1.
If $x$ and $y$ live on a general bounded interval $(0,M)$, then the argument goes through with $alpha<1/M$.
answered Jan 17 at 18:03
Gerhard S.Gerhard S.
1,05029
$endgroup$
$begingroup$
I understand the argument justifying $log(y)-log(x)geqalpha(y-x)$ but could you show it algebraically?
$endgroup$
– Saeed
Jan 17 at 18:53
$begingroup$
@Saeed: $log$ is concave. Hence, $log(x)leqlog(y)+(1/y)(x-y)$. Since $1/ygeq1/M$, one gets $log(y)-log(x)geq(1/y)(y-x)geq(1/M)(y-x)$.
$endgroup$
– Gerhard S.
Jan 17 at 19:27
$begingroup$
Smart way of showing that.
$endgroup$
– Saeed
Jan 17 at 19:32
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3077287%2fhow-to-show-negative-entropy-function-fx-x-logx-is-strongly-convex%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
We have $f'(x)=1+log(x)$. Strict convexity therefore means that there exists a strictly positive $alpha$ such that
$$[log(y)-log(x)](y-x)geqalpha(y-x)^2$$
holds. Without loss of generality I assume $y>x$. Then the above inequality requires that
$$log(y)-log(x)geqalpha(y-x).$$
Although you don't state it, I assume that the variables $x$ and $y$ live on $(0,1)$ (because they are probabilities). Since the slope of the $log$ function on the interval $(0,1)$ is larger than or equal to 1, you can choose any positive $alpha$ smaller than 1.
If $x$ and $y$ live on a general bounded interval $(0,M)$, then the argument goes through with $alpha<1/M$.
answered Jan 17 at 18:03
Gerhard S.Gerhard S.
1,05029
$endgroup$
$begingroup$
I understand the argument justifying $log(y)-log(x)geqalpha(y-x)$ but could you show it algebraically?
$endgroup$
– Saeed
Jan 17 at 18:53
$begingroup$
@Saeed: $log$ is concave. Hence, $log(x)leqlog(y)+(1/y)(x-y)$. Since $1/ygeq1/M$, one gets $log(y)-log(x)geq(1/y)(y-x)geq(1/M)(y-x)$.
$endgroup$
– Gerhard S.
Jan 17 at 19:27
$begingroup$
Smart way of showing that.
$endgroup$
– Saeed
Jan 17 at 19:32
add a comment |
$begingroup$
We have $f'(x)=1+log(x)$. Strict convexity therefore means that there exists a strictly positive $alpha$ such that
$$[log(y)-log(x)](y-x)geqalpha(y-x)^2$$
holds. Without loss of generality I assume $y>x$. Then the above inequality requires that
$$log(y)-log(x)geqalpha(y-x).$$
Although you don't state it, I assume that the variables $x$ and $y$ live on $(0,1)$ (because they are probabilities). Since the slope of the $log$ function on the interval $(0,1)$ is larger than or equal to 1, you can choose any positive $alpha$ smaller than 1.
If $x$ and $y$ live on a general bounded interval $(0,M)$, then the argument goes through with $alpha<1/M$.
answered Jan 17 at 18:03
Gerhard S.Gerhard S.
1,05029
$endgroup$
$begingroup$
I understand the argument justifying $log(y)-log(x)geqalpha(y-x)$ but could you show it algebraically?
$endgroup$
– Saeed
Jan 17 at 18:53
$begingroup$
@Saeed: $log$ is concave. Hence, $log(x)leqlog(y)+(1/y)(x-y)$. Since $1/ygeq1/M$, one gets $log(y)-log(x)geq(1/y)(y-x)geq(1/M)(y-x)$.
$endgroup$
– Gerhard S.
Jan 17 at 19:27
$begingroup$
Smart way of showing that.
$endgroup$
– Saeed
Jan 17 at 19:32
add a comment |
$begingroup$
We have $f'(x)=1+log(x)$. Strict convexity therefore means that there exists a strictly positive $alpha$ such that
$$[log(y)-log(x)](y-x)geqalpha(y-x)^2$$
holds. Without loss of generality I assume $y>x$. Then the above inequality requires that
$$log(y)-log(x)geqalpha(y-x).$$
Although you don't state it, I assume that the variables $x$ and $y$ live on $(0,1)$ (because they are probabilities). Since the slope of the $log$ function on the interval $(0,1)$ is larger than or equal to 1, you can choose any positive $alpha$ smaller than 1.
If $x$ and $y$ live on a general bounded interval $(0,M)$, then the argument goes through with $alpha<1/M$.
answered Jan 17 at 18:03
Gerhard S.Gerhard S.
1,05029
$endgroup$
We have $f'(x)=1+log(x)$. Strict convexity therefore means that there exists a strictly positive $alpha$ such that
$$[log(y)-log(x)](y-x)geqalpha(y-x)^2$$
holds. Without loss of generality I assume $y>x$. Then the above inequality requires that
$$log(y)-log(x)geqalpha(y-x).$$
Although you don't state it, I assume that the variables $x$ and $y$ live on $(0,1)$ (because they are probabilities). Since the slope of the $log$ function on the interval $(0,1)$ is larger than or equal to 1, you can choose any positive $alpha$ smaller than 1.
If $x$ and $y$ live on a general bounded interval $(0,M)$, then the argument goes through with $alpha<1/M$.
answered Jan 17 at 18:03
Gerhard S.Gerhard S.
1,05029
edited Jan 17 at 18:08
answered Jan 17 at 18:03
Gerhard S.Gerhard S.
1,05029
answered Jan 17 at 18:03
Gerhard S.Gerhard S.
1,05029
answered Jan 17 at 18:03
Gerhard S.Gerhard S.
1,05029
1,05029
$begingroup$
I understand the argument justifying $log(y)-log(x)geqalpha(y-x)$ but could you show it algebraically?
$endgroup$
– Saeed
Jan 17 at 18:53
$begingroup$
@Saeed: $log$ is concave. Hence, $log(x)leqlog(y)+(1/y)(x-y)$. Since $1/ygeq1/M$, one gets $log(y)-log(x)geq(1/y)(y-x)geq(1/M)(y-x)$.
$endgroup$
– Gerhard S.
Jan 17 at 19:27
$begingroup$
Smart way of showing that.
$endgroup$
– Saeed
Jan 17 at 19:32
add a comment |
$begingroup$
I understand the argument justifying $log(y)-log(x)geqalpha(y-x)$ but could you show it algebraically?
$endgroup$
– Saeed
Jan 17 at 18:53
$begingroup$
@Saeed: $log$ is concave. Hence, $log(x)leqlog(y)+(1/y)(x-y)$. Since $1/ygeq1/M$, one gets $log(y)-log(x)geq(1/y)(y-x)geq(1/M)(y-x)$.
$endgroup$
– Gerhard S.
Jan 17 at 19:27
$begingroup$
Smart way of showing that.
$endgroup$
– Saeed
Jan 17 at 19:32
$begingroup$
I understand the argument justifying $log(y)-log(x)geqalpha(y-x)$ but could you show it algebraically?
$endgroup$
– Saeed
Jan 17 at 18:53
$begingroup$
I understand the argument justifying $log(y)-log(x)geqalpha(y-x)$ but could you show it algebraically?
$endgroup$
– Saeed
Jan 17 at 18:53
$begingroup$
@Saeed: $log$ is concave. Hence, $log(x)leqlog(y)+(1/y)(x-y)$. Since $1/ygeq1/M$, one gets $log(y)-log(x)geq(1/y)(y-x)geq(1/M)(y-x)$.
$endgroup$
– Gerhard S.
Jan 17 at 19:27
$begingroup$
@Saeed: $log$ is concave. Hence, $log(x)leqlog(y)+(1/y)(x-y)$. Since $1/ygeq1/M$, one gets $log(y)-log(x)geq(1/y)(y-x)geq(1/M)(y-x)$.
$endgroup$
– Gerhard S.
Jan 17 at 19:27
$begingroup$
Smart way of showing that.
$endgroup$
– Saeed
Jan 17 at 19:32
$begingroup$
Smart way of showing that.
$endgroup$
– Saeed
Jan 17 at 19:32
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3077287%2fhow-to-show-negative-entropy-function-fx-x-logx-is-strongly-convex%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
yr1Xhfwp,N4Y e1df,3KrrGXuaXGCEfT4U8wWfOqQfhcHJyORxiGsNvwv01Z XUY6PL,q8qz,o s45
$begingroup$
Why don't you calculate the second derivative?
$endgroup$
– Dr. Sonnhard Graubner
Jan 17 at 17:53
$begingroup$
@Dr. Sonnhard Graubner : Because I want to better understand the definition to proof the general case:math.stackexchange.com/questions/3055187/… and solve it.
$endgroup$
– Saeed
Jan 17 at 17:57
$begingroup$
@Dr. Sonnhard Graubner : Also, the second derivative does not specify $alpha$. Actually, I do not know how to use this fact saying strongly convex implies $nabla^2f(x) succeq alpha I$ for this case to get $alpha$.
$endgroup$
– Saeed
Jan 17 at 18:01
$begingroup$
The case $x=1,,y=1+z>0$ gives $frac{ln(1+z)}{z}gealpha$, which for sufficiently large $z>0$ contradicts any proposed $alpha>0$.
$endgroup$
– J.G.
Jan 17 at 18:27
$begingroup$
@J.G. : I think I should have assumed that $0 leq x leq 1$?
$endgroup$
– Saeed
Jan 17 at 18:32