Papers › Transformer Embeddings of Irregularly Spaced Events and Their Participants

Transformer Embeddings of Irregularly Spaced Events and Their Participants

31 Dec 2021ICLR 2022 4arXiv:2201.00044archive 2025-07-28

Chenghao Yang, Hongyuan Mei, Jason Eisner

The neural Hawkes process (Mei & Eisner, 2017) is a generative model of irregularly spaced sequences of discrete events. To handle complex domains with many event types, Mei et al. (2020a) further consider a setting in which each event in the sequence updates a deductive database of facts (via domain-specific pattern-matching rules); future events are then conditioned on the database contents. They show how to convert such a symbolic system into a neuro-symbolic continuous-time generative model, in which each database fact and the possible event has a time-varying embedding that is derived from its symbolic provenance. In this paper, we modify both models, replacing their recurrent LSTM-based architectures with flatter attention-based architectures (Vaswani et al., 2017), which are simpler and more parallelizable. This does not appear to hurt our accuracy, which is comparable to or better than that of the original models as well as (where applicable) previous attention-based methods (Zuo et al., 2020; Zhang et al., 2020a).

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

yangalan123/anhp-andtt officialmentioned in papermentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Self-Attentive Hawkes Processes (arXiv:1907.07561; that paper's own run record: 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified), Transformer Hawkes Process (arXiv:2002.09291; that paper's own run record: 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified), HYPRO: A Hybridly Normalized Probabilistic Model for Long-Horizon Prediction of Event Sequences (arXiv:2210.01753; that paper's own run record: 18 ran (of which 6 constructed an object rather than computing a result; 17 with no instrument failure: 0 honoured, 0 violated, 17 with no contract checked; 1 where Syntology's instrument failed) · 6 unverified) and 1 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report
ivan-chai/hotpp-benchmark pytorchApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to HYPRO: A Hybridly Normalized Probabilistic Model for Long-Horizon Prediction of Event Sequences (arXiv:2210.01753; that paper's own run record: 18 ran (of which 6 constructed an object rather than computing a result; 17 with no instrument failure: 0 honoured, 0 violated, 17 with no contract checked; 1 where Syntology's instrument failed) · 6 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

Absolute Position EncodingsAdamAttentionBPEDense ConnectionsDropoutLSTMLabel SmoothingLayer NormalizationLinear LayerMEIMulti-Head AttentionPosition-Wise Feed-Forward LayerResidual ConnectionSigmoid ActivationSoftmaxTanh ActivationTransformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections