Skip to content

add adaptive methods for convex problems - #107

Open
aldma wants to merge 8 commits into
JuliaFirstOrder:masterfrom
aldma:adaptive-methods
Open

aldma wants to merge 8 commits into
JuliaFirstOrder:masterfrom
aldma:adaptive-methods

Conversation

@aldma

@aldma aldma commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator

This PR adds the adaptive methods for convex problems developed in Latafat, Themelis, Stella, Patrinos, "Adaptive proximal algorithms for convex optimization under local Lipschitz continuity of the gradient," Mathematical Programming, vol. 213, pp. 433–471 (2025), whose accompanying code is at https://github.com/pylat/adaptive-proximal-algorithms.

@aldma aldma mentioned this pull request Oct 29, 2026
@aldma aldma mentioned this pull request Nov 12, 2026
Comment thread src/algorithms/adapgm.jl Resolved
Comment thread src/algorithms/adapgm.jl Resolved
lostella
lostella previously approved these changes Nov 16, 2026
@aldma

aldma commented Nov 16, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks @lostella ! Any comments/suggestions on adapdm.jl? @pylat
Soon I will try to look into allocations and performance too

@aldma

aldma commented Nov 16, 2026

Copy link
Copy Markdown
Collaborator Author

Question on the stopping criterion for AdaPDM:
the current termination condition is norm(FPR_x, Inf) + norm(FPR_y, Inf) <= tol where FPR_x = x-xold.
Other algorithms in the package typically divide the residual by the stepsize, for example AdaPGM has norm(res, Inf) / gamma <= tol, with res = x - xold. Does it make more sense to consider here the scaled version?

@pylat

pylat commented Nov 16, 2026

Copy link
Copy Markdown
Member

Question on the stopping criterion for AdaPDM: the current termination condition is norm(FPR_x, Inf) + norm(FPR_y, Inf) <= tol where FPR_x = x-xold. Other algorithms in the package typically divide the residual by the stepsize, for example AdaPGM has norm(res, Inf) / gamma <= tol, with res = x - xold. Does it make more sense to consider here the scaled version?

Yes, the termination condition is not accurate. It should be as in https://github.com/pylat/adaptive-proximal-algorithms/blob/2eb4059c6dbf7a1574073f854b3b4bc9bc922587/src/AdaProx.jl#L348

It constructs v^k in T(z^k) so it is a proper surrogate for distance(0, T(z^k)).

@aldma

aldma commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks @pylat for pointing me to https://github.com/pylat/adaptive-proximal-algorithms. I have changed the residuals and termination conditions accordingly in AdaPDM. Could you please check it? Then, @lostella could give a last read before merging. Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants