{ "access": { "embargo": { "active": false, "reason": null }, "files": "public", "record": "public", "status": "open" }, "created": "2023-01-15T11:15:55.485258+00:00", "custom_fields": { "journal:journal": { "issue": "2", "pages": "394-423", "title": "SIAM Journal on Optimization", "volume": "33" } }, "deletion_status": { "is_deleted": false, "status": "P" }, "files": { "count": 1, "enabled": true, "entries": { "Jakovetic_etal_SIAM2022.pdf": { "checksum": "md5:d0526628ef5e32b9ba23b626b2d37a16", "ext": "pdf", "id": "2c6483e1-df8e-425f-8660-1308ae4e3fd2", "key": "Jakovetic_etal_SIAM2022.pdf", "metadata": null, "mimetype": "application/pdf", "size": 2095470 } }, "order": [], "total_bytes": 2095470 }, "id": "7538446", "is_draft": false, "is_published": true, "links": { "access": "https://zenodo.org/api/records/7538446/access", "access_links": "https://zenodo.org/api/records/7538446/access/links", "access_request": "https://zenodo.org/api/records/7538446/access/request", "access_users": "https://zenodo.org/api/records/7538446/access/users", "archive": "https://zenodo.org/api/records/7538446/files-archive", "archive_media": "https://zenodo.org/api/records/7538446/media-files-archive", "communities": "https://zenodo.org/api/records/7538446/communities", "communities-suggestions": "https://zenodo.org/api/records/7538446/communities-suggestions", "doi": "https://doi.org/10.5281/zenodo.7538446", "draft": "https://zenodo.org/api/records/7538446/draft", "files": "https://zenodo.org/api/records/7538446/files", "latest": "https://zenodo.org/api/records/7538446/versions/latest", "latest_html": "https://zenodo.org/records/7538446/latest", "media_files": "https://zenodo.org/api/records/7538446/media-files", "parent": "https://zenodo.org/api/records/7538445", "parent_doi": "https://zenodo.org/doi/10.5281/zenodo.7538445", "parent_html": "https://zenodo.org/records/7538445", "requests": "https://zenodo.org/api/records/7538446/requests", "reserve_doi": "https://zenodo.org/api/records/7538446/draft/pids/doi", "self": "https://zenodo.org/api/records/7538446", "self_doi": "https://zenodo.org/doi/10.5281/zenodo.7538446", "self_html": "https://zenodo.org/records/7538446", "self_iiif_manifest": "https://zenodo.org/api/iiif/record:7538446/manifest", "self_iiif_sequence": "https://zenodo.org/api/iiif/record:7538446/sequence/default", "versions": "https://zenodo.org/api/records/7538446/versions" }, "media_files": { "count": 0, "enabled": false, "entries": {}, "order": [], "total_bytes": 0 }, "metadata": { "creators": [ { "affiliations": [ { "name": "University of Novi Sad, Faculty of Sciences, Department of Mathematics and Informatics" } ], "person_or_org": { "family_name": "Dusan Jakovetic", "name": "Dusan Jakovetic", "type": "personal" } }, { "affiliations": [ { "name": "niversity of Novi Sad, Faculty of Technical Sciences, Department of Power, Electronic and Communication Engineering" } ], "person_or_org": { "family_name": "Dragana Bajovic", "name": "Dragana Bajovic", "type": "personal" } }, { "affiliations": [ { "name": "Amazon Alexa AI" } ], "person_or_org": { "family_name": "Anit Kumar Sahu", "name": "Anit Kumar Sahu", "type": "personal" } }, { "affiliations": [ { "name": "Department of Electrical and Computer Engineering, Carnegie Mellon University" } ], "person_or_org": { "family_name": "Soummya Kar", "name": "Soummya Kar", "type": "personal" } }, { "affiliations": [ { "name": "University of Novi Sad, Faculty of Sciences, Department of Mathematics and Informatics" } ], "person_or_org": { "family_name": "Nemanja Milosevic", "name": "Nemanja Milosevic", "type": "personal" } }, { "affiliations": [ { "name": "University of Novi Sad, Faculty of Sciences, Department of Mathematics and Informatics" } ], "person_or_org": { "family_name": "Dusan Stamenkovic", "name": "Dusan Stamenkovic", "type": "personal" } } ], "description": "
We introduce a general framework for nonlinear stochastic gradient descent (SGD) for the scenarios when gradient noise exhibits heavy tails. The proposed framework subsumes several popular nonlinearity choices, like clipped, normalized, signed or quantized gradient, but we also consider novel nonlinearity choices. We establish for the considered class of methods strong convergence guarantees assuming a strongly convex cost function with Lipschitz continuous gradients under very general assumptions on the gradient noise. Most notably, we show that, for a nonlinearity with bounded outputs and for the gradient noise that may not have finite moments of order greater than one, the nonlinear SGD’s mean squared error (MSE), or equivalently, the expected cost function’s optimality gap, converges to zero at rate O(1/tζ ), ζ ∈ (0, 1). In contrast, for the same noise setting, the linear SGD generates a sequence with unbounded variances. Furthermore, for general nonlinearities that can be decoupled component wise and a class of joint nonlinearities, we show that the nonlinear SGD asymptotically (locally) achieves a O(1/t) rate in the weak convergence sense and explicitly quantify the corresponding asymptotic variance. Experiments show that, while our framework is more general than existing studies of SGD under heavy-tail noise, several easy-to-implement nonlinearities from our framework are competitive with state-of-the-art alternatives on real data sets with heavy tail noises.
", "funding": [ { "award": { "acronym": "MARVEL", "id": "00k4n6c32::957337", "identifiers": [ { "identifier": "https://cordis.europa.eu/projects/957337", "scheme": "url" } ], "number": "957337", "program": "H2020", "title": { "en": "Multimodal Extreme Scale Data Analytics for Smart Cities Environments" } }, "funder": { "id": "00k4n6c32", "name": "European Commission" } } ], "identifiers": [ { "identifier": "10.1137/21M145896X", "scheme": "doi" } ], "languages": [ { "id": "eng", "title": { "en": "English" } } ], "publication_date": "2023-01-15", "publisher": "Zenodo", "resource_type": { "id": "publication-journal", "title": { "de": "Zeitschrift", "en": "Journal" } }, "rights": [ { "description": { "en": "The Creative Commons Attribution license allows re-distribution and re-use of a licensed work on the condition that the creator is appropriately credited." }, "icon": "cc-by-icon", "id": "cc-by-4.0", "props": { "scheme": "spdx", "url": "https://creativecommons.org/licenses/by/4.0/legalcode" }, "title": { "en": "Creative Commons Attribution 4.0 International" } } ], "subjects": [ { "subject": "Stochastic optimization" }, { "subject": "stochastic gradient descent" }, { "subject": "nonlinear mapping" }, { "subject": "heavy-tail noise" }, { "subject": "convergence rate" }, { "subject": "mean square analysis" }, { "subject": "asymptotic normality" }, { "subject": "stochastic approximation" } ], "title": "Non-linear Gradient Mappings and Stochastic Optimization: a General Framework with Applications to Heavy-Tail Noise" }, "parent": { "access": { "owned_by": { "user": 212013 }, "settings": { "accept_conditions_text": null, "allow_guest_requests": false, "allow_user_requests": false, "secret_link_expiration": 0 } }, "communities": { "default": "cd44bdb7-1599-4cdc-979e-922fa0973a41", "entries": [ { "access": { "member_policy": "open", "members_visibility": "public", "record_policy": "open", "review_policy": "open", "visibility": "public" }, "children": { "allow": false }, "created": "2021-04-10T22:22:48.944591+00:00", "custom_fields": {}, "deletion_status": { "is_deleted": false, "status": "P" }, "id": "cd44bdb7-1599-4cdc-979e-922fa0973a41", "links": {}, "metadata": { "curation_policy": "", "page": "The MARVEL project is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No 957337.
\r\n\r\nMARVEL aspires the convergence of a set of technologies in the areas of AI, analytics, multimodal perception, software engineering, HPC as part of an Edge-Fog-Cloud Computing Continuum paradigm that goes beyond traditional Big Data, conventional architectures heavily capitalizing on distributed resources andheterogeneous data sources in smart city environments, while implementing privacy preservation techniques at all data modalities and at all levels of its architecture.
\r\n\r\nThe ultimate aim is to support data-driven real-time application workflows and decision making in modern cities, showcasing the potential to address societal challenges very effectively, from increasing public safety and security to analysingtraffic flows and traffic behaviour in the cities of Trento and Malta.
\r\n\r\nhttps://www.marvel-project.eu
", "title": "H2020 MARVEL Project - Multimodal Extreme Scale Data Analytics for Smart Cities Environments" }, "revision_id": 0, "slug": "marvel_project", "updated": "2021-04-10T22:22:49.055714+00:00" }, { "access": { "member_policy": "open", "members_visibility": "restricted", "record_policy": "open", "review_policy": "closed", "visibility": "public" }, "children": { "allow": true }, "created": "2022-11-23T15:53:29.436323+00:00", "custom_fields": {}, "deletion_status": { "is_deleted": false, "status": "P" }, "id": "f0a8b890-f97a-4eb2-9eac-8b8a712d3a6c", "links": {}, "metadata": { "curation_policy": "The EU Open Research Repository serves as a repository for research outputs (data, software, posters, presentations, publications, etc) which have been funded under an EU research funding programme such as Horizon Europe, Euratom or earlier Framework Programmes.
\nThe community is managed by CERN on behalf of the European Commission.
\nZenodo’s general policies and Terms of Use apply to all content.
\nThe EU Open Research Repository accepts all digital research objects which is a research output stemming from one of EU’s research and innovation funding programmes. The funding programmes currently include:
\nHorizon Europe (including ERC, MSCA), earlier Framework Programmes (eg Horizon 2020) as well as Euratom.
\nIn line with the principle as open as possible, as closed as necessary both public and restricted content is accepted. See note on how Zenodo handles restricted content.
\nEU programme beneficiaries are eligible to submit content to the community. The community supports three types of content submissions:
\nSubmission via an EU Project Community (through user interface or programmatic APIs).
\nSubmission directly to the EU Open Research Repository.
\nAutomated harvesting from existing Zenodo content.
\nA representative of an EU project may request an EU Project Community and invite other project participants as members of the community. The project community is linked to one or more European Commission grants. All records in the project community are automatically integrated into the EU Open Research Repository immediately upon acceptance into the project community.
\nAny user may submit a record directly to the EU Open Research Repository. The submission will be moderated by Zenodo staff for compliance with the minimal required metadata requirements and its correctness.
\nRecords found among Zenodo’s existing content will on a regular basis automatically be integrated if they are found to comply with the requirements. The submissions through this method are integrated into the EU Open Research Repository with delay in a fully automated way.
\nRecords in the EU Open Research Repository are required to comply with the following minimal metadata requirements:
\nVisibility: Both public and restricted (with or without embargo and/or access request)
\nResource types: All resource types.
\nLicenses: Public and embargoed records MUST specify a license.
\nFunding information: Records MUST specify at least one grant from the European Commission.
\nCreators: Creators SHOULD be identified with a persistent identifier (e.g. ORCID, GND, …), and affiliations SHOULD be identified with a persistent identifier (e.g. ROR, ISNI, …)
\nSubjects: Records SHOULD specify one or more fields of science from the European Science Vocabulary.
\nAll submissions will undergo automated curation checks for compliance with the policy. Submissions through project communities are reviewed by the project community. Submission directly to the EU Open Research Repository is reviewed by Zenodo staff.
\nCommunity curators may at any point edit metadata of the records in the community without notice through human or automated processing. The curators may at their sole discretion remove records from the community that are deemed not to comply with the content and curation policy or which are deemed of insufficient quality.
\nThe content and curation policy is subject to change by the community owner at any time and without notice, other than through updating this page.
", "description": "Open repository for EU-funded research outputs from Horizon Europe, Euratom and earlier Framework Programmes.", "organizations": [ { "id": "00k4n6c32" } ], "page": "The EU Open Research Repository is a Zenodo-community dedicated to fostering open science and enhancing the visibility and accessibility of research outputs funded by the European Union. The community is managed by CERN on behalf of the European Commission.
\nThe mission of the repository is to support the implementation of the EU's open science policy, providing a trusted and comprehensive space for researchers to share their research outputs such as data, software, reports, presentations, posters and more. The EU Open Research Repository simplifies the process of complying with open science requirements, ensuring that research outputs from Horizon Europe, Euratom, and earlier Framework Programmes are freely accessible, thereby accelerating scientific discovery and innovation.
\nThe EU Open Research Repository serves as a complementary platform to the Open Research Europe (ORE) publishing platform. Open Research Europe focuses on providing a publishing venue for peer-reviewed articles, ensuring that research meets rigorous academic standards. The EU Open Research Repository provides a space for all the other research outputs including data sets, software, posters, and presentations that are out of scope for ORE. This holistic approach enables researchers to not only publish their findings but also share the underlying data and materials that support their work, fostering transparency and reproducibility in the scientific process.
\nCurrently in its pilot phase and set to be fully operational during autumn 2024, the EU Open Research Repository is constantly evolving. Efforts are committed to integrating cutting-edge features, including automated curation checks and FAIR (Findable, Accessible, Interoperable, and Reusable) assistance, to further support the research community. The goal is to provide researchers with a simple goto solution for making their publicly funded research open and as FAIR as possible.
\nThe EU Open Research Repository is funded by the European Union under grant agreement no. 101122956(HORIZON-ZEN). For more information about the project see https://about.zenodo.org/projects/horizon-zen/.
", "title": "EU Open Research Repository (Pilot)", "type": { "id": "organization" }, "website": "https://research-and-innovation.ec.europa.eu" }, "revision_id": 16, "slug": "eu", "theme": { "brand": "horizon", "enabled": true, "style": { "font": { "family": "Arial, sans-serif", "size": "16px", "weight": 600 }, "mainHeaderBackgroundColor": "#FFFFFF", "primaryColor": "#004494", "primaryTextColor": "#FFFFFF", "secondaryColor": "#FFD617", "secondaryTextColor": "#000000", "tertiaryColor": "#e3eefd", "tertiaryTextColor": "#1c5694" } }, "updated": "2024-03-20T06:47:47.577483+00:00" } ], "ids": [ "cd44bdb7-1599-4cdc-979e-922fa0973a41", "f0a8b890-f97a-4eb2-9eac-8b8a712d3a6c" ] }, "id": "7538445", "pids": { "doi": { "client": "datacite", "identifier": "10.5281/zenodo.7538445", "provider": "datacite" } } }, "pids": { "doi": { "client": "datacite", "identifier": "10.5281/zenodo.7538446", "provider": "datacite" }, "oai": { "identifier": "oai:zenodo.org:7538446", "provider": "oai" } }, "revision_id": 9, "stats": { "all_versions": { "data_volume": 108964440.0, "downloads": 52, "unique_downloads": 45, "unique_views": 69, "views": 78 }, "this_version": { "data_volume": 108964440.0, "downloads": 52, "unique_downloads": 45, "unique_views": 69, "views": 78 } }, "status": "published", "updated": "2024-01-02T11:30:37.327748+00:00", "versions": { "index": 1, "is_latest": true } }