Combining embedded accelerometers with computer vision for recognizing food preparation activities

S. Stein, S.J. McKenna

    Research output: Chapter in Book/Report/Conference proceedingChapter

    57 Citations (Scopus)

    Abstract

    This paper introduces a publicly available dataset of complex activities that involve manipulative gestures. The dataset captures people preparing mixed salads and contains more than 4.5 hours of accelerometer and RGB-D video data, detailed annotations, and an evaluation protocol for comparison of activity recognition algorithms. Providing baseline results for one possible activity recognition task, this paper further investigates modality fusion methods at different stages of the recognition pipeline: (i) prior to feature extraction through accelerometer localization, (ii) at feature level via feature concatenation, and (iii) at classification level by combining classifier outputs. Empirical evaluation shows that fusing in- formation captured by these sensor types can considerably improve recognition performance.
    Original languageEnglish
    Title of host publicationUbiComp 2013 - Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing
    Pages729-738
    Number of pages10
    DOIs
    Publication statusPublished - 1 Jan 2013

    Fingerprint

    Accelerometers
    Computer vision
    Feature extraction
    Classifiers
    Fusion reactions
    Pipelines
    Sensors

    Cite this

    Stein, S., & McKenna, S. J. (2013). Combining embedded accelerometers with computer vision for recognizing food preparation activities. In UbiComp 2013 - Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (pp. 729-738) https://doi.org/10.1145/2493432.2493482
    Stein, S. ; McKenna, S.J. / Combining embedded accelerometers with computer vision for recognizing food preparation activities. UbiComp 2013 - Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 2013. pp. 729-738
    @inbook{6502776a5fea4e1aa8dc218a226a3e83,
    title = "Combining embedded accelerometers with computer vision for recognizing food preparation activities",
    abstract = "This paper introduces a publicly available dataset of complex activities that involve manipulative gestures. The dataset captures people preparing mixed salads and contains more than 4.5 hours of accelerometer and RGB-D video data, detailed annotations, and an evaluation protocol for comparison of activity recognition algorithms. Providing baseline results for one possible activity recognition task, this paper further investigates modality fusion methods at different stages of the recognition pipeline: (i) prior to feature extraction through accelerometer localization, (ii) at feature level via feature concatenation, and (iii) at classification level by combining classifier outputs. Empirical evaluation shows that fusing in- formation captured by these sensor types can considerably improve recognition performance.",
    author = "S. Stein and S.J. McKenna",
    year = "2013",
    month = "1",
    day = "1",
    doi = "10.1145/2493432.2493482",
    language = "English",
    isbn = "9781450317702",
    pages = "729--738",
    booktitle = "UbiComp 2013 - Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing",

    }

    Stein, S & McKenna, SJ 2013, Combining embedded accelerometers with computer vision for recognizing food preparation activities. in UbiComp 2013 - Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing. pp. 729-738. https://doi.org/10.1145/2493432.2493482

    Combining embedded accelerometers with computer vision for recognizing food preparation activities. / Stein, S.; McKenna, S.J.

    UbiComp 2013 - Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 2013. p. 729-738.

    Research output: Chapter in Book/Report/Conference proceedingChapter

    TY - CHAP

    T1 - Combining embedded accelerometers with computer vision for recognizing food preparation activities

    AU - Stein, S.

    AU - McKenna, S.J.

    PY - 2013/1/1

    Y1 - 2013/1/1

    N2 - This paper introduces a publicly available dataset of complex activities that involve manipulative gestures. The dataset captures people preparing mixed salads and contains more than 4.5 hours of accelerometer and RGB-D video data, detailed annotations, and an evaluation protocol for comparison of activity recognition algorithms. Providing baseline results for one possible activity recognition task, this paper further investigates modality fusion methods at different stages of the recognition pipeline: (i) prior to feature extraction through accelerometer localization, (ii) at feature level via feature concatenation, and (iii) at classification level by combining classifier outputs. Empirical evaluation shows that fusing in- formation captured by these sensor types can considerably improve recognition performance.

    AB - This paper introduces a publicly available dataset of complex activities that involve manipulative gestures. The dataset captures people preparing mixed salads and contains more than 4.5 hours of accelerometer and RGB-D video data, detailed annotations, and an evaluation protocol for comparison of activity recognition algorithms. Providing baseline results for one possible activity recognition task, this paper further investigates modality fusion methods at different stages of the recognition pipeline: (i) prior to feature extraction through accelerometer localization, (ii) at feature level via feature concatenation, and (iii) at classification level by combining classifier outputs. Empirical evaluation shows that fusing in- formation captured by these sensor types can considerably improve recognition performance.

    UR - http://www.scopus.com/inward/record.url?scp=84885201691&partnerID=8YFLogxK

    U2 - 10.1145/2493432.2493482

    DO - 10.1145/2493432.2493482

    M3 - Chapter

    AN - SCOPUS:84885201691

    SN - 9781450317702

    SP - 729

    EP - 738

    BT - UbiComp 2013 - Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing

    ER -

    Stein S, McKenna SJ. Combining embedded accelerometers with computer vision for recognizing food preparation activities. In UbiComp 2013 - Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 2013. p. 729-738 https://doi.org/10.1145/2493432.2493482