News

TelConf 08.06.2011 - Introduction of BUT to GLocal partners

= People around =
* ?? - Thesaloniki
* Francesco - Trento
* Denis - AfP
* Claudia - ISOCO
* BUT: PavelS, Stefan, Honza

= Honza presents the BUT group =
* LVCSR, keyword spotting, spoken term detection
* English, reasonable quality audio.

= Scenarios =
* wil target more the professional scenario.
* AfP - all videos in GLOCAL repository are in ENglish and have double audio track.
* Denis - reporting on mobile phone from an event - is it is the user, possible.

* anotation is better ... but at some point there'll be a search.
* Francesco - audio should become part of the model. Prove or deny that an audio is or is not relevant to an event.
* Honza also pushes the kwds.

== what we can do ==
* Royal Wedding - very useful, people agreed on this in Hannover meeting.

== what is going in video ==
*Vasilis - visual <b>concept detection</b>, physical objects, people... confidence scores
*ID of more complex concept from elementary events
*ID of relevant parts of video correlating with a concept.
Ways to introduce audio info:
*non-speech audio signals - elementary audio events. BUT not able to do this.
*textual transcripts.
*for example for royal wedding, can detect people based on a list.
*not exactly given people, but more generic concepts. But doable as well.
*or use the audio to verify the concept - just by searching list of keywords inside.
Fusing of info:
*event-based
*whole video.

Common Data

Link to the interface: http://www.dfki.de/glocal/latest/

 

Year 2 integration plan (Carlos):

 

Event types to be considered:

  • Soccer/Football - World Championship 2010
  • Social events (Concerts, etc.)
  • Wedding

 

XML schemas and conventions

  • Deliverables in WP1 are found here: http://www.glocal-project.eu/wiki/wp-1-deliverables
  • Deliverable 1.1 defines the model for the events and entities representation in XML, there is some more definition in D1.2
  • URI have topoint to metadata (XML formatted) about the represented entity

 

Data sets

  • CERTH: Flickr images with geo-location for cities (4 cities)
  • UNITN:
    • 1 million photos with geo-location (photo ids and geo tags)
    • toy data set for the world cup scenario (created manually for the review)
  • LUH: Flickr, Upcoming, LastFM 
    • 13k events from Upcoming with time, place, users/attendees, tags, description, geo-location, etc.
    • 316k Flickr images linked to Upcoming events using Flickr machine tags
    • 3m Flickr images with tags, time, description, users, geo-location (~25% of the photos)
    • concert events in LastFM
  • INRIA (for WP2)
    • The dataset we use was constructed from the corpus introduced by Troncy and al. [1] for the general evaluation of event-centric indexing approaches. This corpus mainly contained events and media descriptionscand was originally created from three large public event directories (last.fm, eventful and upcoming). In our case, we only used it to define a set of Flickr images labeled with last.fm tags, i.e unique identifiers of music events such as concerts, festivals, etc. The images themselves were not provided in the data and we had to crawl them resulting in some missing images. We used the EXIF creation date field of the pictures to generate the time metadata used in our method. About 50% only of the crawled images had such valid EXIF.
      Finally, we have a dataset of 828902 images with 10257 users / 34, 034 distinct LastFM events / 41, 294 event's record (=1 user@1 event).
      [1] R. Troncy, B. Malocha, and A. T. S. Fialho. Linking events with media. In Proceedings of the 6th International Conference on Semantic Systems, 2010
    • http://www.eurecom.fr/~troncy/ldtc2010/2010-06-15-N3_Events.zip
    • http://www.eurecom.fr/~troncy/ldtc2010/2010-06-16-N3_Flickr.zip

 

Software modules in WP3

This page should contain all modules created (or planned to be created) in WP3. Please list their descriptions, endpoints, inputs, outputs, etc.

 

Contact information of partners involved in the creation of the modules and available for questions/support.

 

 

 Users  Repository

Enables user- and event-specific metadata retrieval.

WADL:  http://pharos.l3s.uni-hannover.de:8090/application.wadl

Methods - USERS:

  • Get User by ID 

Method for retrieving the user details for the specified user id 

IN: {ID} = user ID in the repository [given as path parameter] 

OUT: Information (both generated in GLOCAL and crawled from external sources, e.g. Flickr, LastFM, Upcoming) about the user and similar users  

Example: http://pharos.l3s.uni-hannover.de:8090/users/11668 

  • Select User Connections 

Method for retrieving the user friendship and neighbor connections for the specified user id 

IN: {ID} = user ID in the repository [given as path parameter] 

OUT: Information about the user‟s friends and neighbors (similar users) 

Example: http://pharos.l3s.uni-hannover.de:8090/users/selectUserConnections/11668 

Methods - EVENTS:

  • List Events - /events (LUH)

Lists 30 events from the database - for testing purposes for now

IN: null

OUT:  list of event URIs and labels

Example:  http://pharos.l3s.uni-hannover.de:8090/events

  • Similar Events by Tags - /events/tags (LUH)

Lists events matching the given list of tags. Offline computation required first from "User/Event Analyzer - Identify Candidate Events for Media"

IN: inTags=<list of tags separated by ":">

OUT: list of Upcoming IDs of the matching events (eg the event info from the Upcoming ID 1268524 can be retrieved from http://upcoming.yahoo.com/event/1268524)

Example: http://pharos.l3s.uni-hannover.de:8090/events/tags/?inTags=Round%20Table...

  • Event by ID - /events/{eventid} (LUH)

Gets the information about a given event ID from the database

IN: event ID given in path, not as parameter

OUT:  Informaion (from Upcoming) about the event; similar events and event media items

Example: http://pharos.l3s.uni-hannover.de:8090/events/2904719

  • DB Query - /events/search (LUH)

Query the events DB by user selected queries

IN: query=<SQL clauses separated by ":" in the format "field=value">

OUT:  list of event URIs and labels matching the query

Examples: http://pharos.l3s.uni-hannover.de:8090/events/search?query=name=New%20Yo...
http://pharos.l3s.uni-hannover.de:8090/events/search?query=UpcomingID=10...

 

Importer (LUH)

Imports data from different online sources into the Users Repository

Data sources:

  • Flickr (image metadata: users, photos, tags, title, description)
  • LastFM (social events: tags, users)
  • Upcoming (event metadata: users / attendees, tags, time, place)

Connections from Flickr to Upcoming or LastFM can be done through machine tags on Flickr of the form "upcoming:*" or "lastfm:*"

 

User/Event Analyzer

Offline analysis of users and communities centered around events.

Methods:

  • Identify Event Communities (LUH)

Analyzes users’ interaction and user generated metadata to mine implicit event-communites.

  • Identify Highly Similar Events (LUH)

Near duplicate detection for events.

  • Identify Candidate Events for Media (LUH)

Given set of tags describing a medium, find photos in the DB with similar tags and return the event where most found photos are part of. Online interface is "Users Repository - Similar Events by Tags".

 

Event Provenance Service [EPS] (iSOCO)

It aims at publishing provenance-related metadata about event life-cycle (e.g. creation, modification...) including the algorithm/component through which each new event of data is created/manipulated (e.g. manual creation, component X ...). This annotation of events is based on semantic provenance vocabularies.

Methods:

  • Set provenance data for a given event
  • Provenance data by event ID: Get provenance information about an event
  • Search: query the triple store

 

New Event Detection (CERTH)

Given a large set of tagged media with geo-location information identify clusters of media where each cluster can be considered to represent a new event. (see http://www.clusttour.gr/glocal).

Methods:

Media Representation 

  • Visual-based representation

The image is represented as a vector of visual words.

In: MediaID, representationType

Out: Occurrence distribution of 500 pre-specified visual words in the image.

e.g., http://160.40.50.139:8085/compareMediaWP3/extractRepresentation?mediaId=...

  • Tag-based representation

The image is represented as a vector of tag words.

In: MediaID, , representationType

Out: Occurrence distribution of 1000 pre-specified tag words in the image.

e.g., http://160.40.50.139:8085/compareMediaWP3/extractRepresentation?mediaId=...

  • Media-based representation

The image is represented as a vector of latent words.

In: MediaID, , representationType

Out: probability distribution of 50 pre-specified latent words.

e.g., http://160.40.50.139:8085/compareMediaWP3/extractRepresentation?mediaId=...

Measure similarity distance between media

  • Visual-based similarity

Measures the euclidean, cosine, l1norm and jaccard distance between the visual-based representations of two media.

In: MediaID1, MediaID2, distanceType

Out: Visual similarity distance between MediaID1, MediaID2

e.g.,  http://160.40.50.139:8085/compareMediaWP3/measureVisualDistance?mediaId1=481447114&mediaId2=503120923&distanceType=cosine

  • Tag-based similarity

Measures the euclidean, cosine, l1norm and jaccard distance between the tag-based representations of two media.

In: MediaID1, MediaID2, distanceType

Out: Tag similarity distance between media MediaID1, MediaID2

e.g., http://160.40.50.139:8085/compareMediaWP3/measureTagDistance?mediaId1=48...

  • Media-based similarity

Measures the euclidean, cosine and linorm distance between the media-based representations of two media

In: MediaID1, MediaID2, distanceType

Out: Media similarity distance between media MediaID1, MediaID2

e.g., http://160.40.50.139:8085/compareMediaWP3/measureMediaDistance?mediaId1=...

Group media into clusters & expose event-media relations

Offline process:

  • Perform a clustering procedure that organizes media into groups based on their similarity distances.
  • Use the media geo-location information to filter out non-interesting clusters. 
  • Extract a rough description for each of the identified clusters based on its most characteristic tags.

Expose event-media relations through rest services: (Tentative, subject to changes)

  • Return a list of all identified clusters. 
  • Return a list of clusters that contain the input tag.
  • Return a list of clusters contained in the provided bounding rectangle.
  • Return a list of clusters associated with the particular place.
  • Filter the photos of the clusters based on the usage license.
  • Return a list of clusters that are contained between the provided time interval (e.g., begin_date and end_date). 

 

Event Matching (UNITN)

  • based on the metadata repository: http://glocal-dkm.science.unitn.it/services/api_help.html
  • Get a similarity score between two events (plans to get a more structured output)
  • Get matching Users for one User -- find a list of users that have similar events to a given user
  • Get matching Events for one Event -- find a list of events that are similar to a given user

 

User Interface (DFKI) (only WP3-related features)

Available at:

  • Last release (Y1-Review Meeting / version 2): http://www.dfki.de/glocal/GlocalUiPg2
  • Latest inoffical version: http://www.dfki.de/glocal/latest

Suggest Events for Media

  • A user can import all media of a Flickr account and the system will visualize events for each media item. The suggestions themselves are retrieved by using service The user can choose which media items she would like to import and one event for each medium.
  • IN: a set of media from Flickr (each Flickr medium is referred to by its square thumbnail URL, e.g.: http://farm6.static.flickr.com/5219/5384616740_d31f77c7bc_s.jpg)
  • OUT: a set of medium-event-links

Suggest Media Alignment to a Given Set of Events

  • While browsing or organizing an event, the user can choose to add new media to that particular event and its sub-events. The for each given medium, the systems retrieves suitable events filtered by the currently visible event and shows a temporary represenation for the medium aligned to that event. The user will then decided whether the suggestions are correct or wrong, finally, when all suggestions are confirmed or rejected (the user might also accept or refuse all at once), the alignment becomes permanant.
  • The server-side services are dummy implementations so far
  • IN: set of media
  • OUT: a set of updated events with media aligned to them

 

Connected Web Services

  • Retrieve event description (XML) by URI ("Event by ID - /events/{eventid} (LUH)")
  • Search events by keywords ("DB Query - /events/search (LUH)")
  • Retrieve similar events ("Similar Events by Tags - /events/tags (LUH)")
  • Get suitable events for a given set of media (Not mentioned in the description above... (?))

 

 

Services to Be Connected

  • Retrieve medium description (XML) by URI (the medium URI can not be used as url for the XML so far)
  • Search events by keywords ("DB Query - /events/search (LUH)")
  • Get comments by URI (as soon as there are comments in the event XML of L3S - see UniTN service)
  • Services of the user repository like getFriendsOf(userId) or suggestFriendsFor(userId)

Anaheim (US): report from conference and workshop activity

Please find attached the the report made about WP7 dissemination 
activity for Andrea de Polo trip to Anaheim, US during the PMA 
http://pmai.org and the I3A http://i3a.org Imaging Standard events/conferences 
and workshops last February 2010. 

Barcelona second plenary meeting

Powerpoint presentation of the activity of the consortium partners in the first 6 months of the project.

Attached images from the consortium meeting hold at the Yahoo! location. Thank you so much Vanessa for your excellent hospitality!

WP7 - Dissemination and Exploitation

WP7 consortium presentations, dissemination activities, including supporting evidences of the events such as videos, pictures from the conferences, proceedings, call for papers, agenda of the consortium planned events to be attended in the near future.

GLOCAL Kick-Off Meeting

The GLOCAL kick-off meeting was held on December 16th–18th in Madonna di Campiglio, Trentino, Italy.

Download the Project Brochure here.

You can follow us on:

RSS Feed

 

 

Links to recomended projects and partners


Syndicate content