CSE 133

CSE 133 Discussion 4/14/99


  • MP2


    • - questions, problems
    - reminder: due date: 4/15/99 Midnight.

  • Frontiers of Information Retrieval - Retrieving and Querying SemistructuredData.

    • 1. What's semistructed data, and what is its relation to CSE133?

  • def: data on a page is semistructured if the data can be located using a concise, formal grammar. [Knoblock]

    Example:

    <html>
<title>Some Country Codes </title><body>
<b>Congo </b> <i> 242 </i > <br >
<b>Egypt </b > <i> 20 </i> <br>
<b>Belize </b> <i> 501 </i> <br>
<b>Spain </b> <i> 34 </i> <br>
</body>
</html>
    The page has some structure- it is a set of tuples, with two attributes per tuple: { Country, Area_Code }.

  • More features of semistructured data:
    - structure may be irregular or incomplete (i.e. attributes missing)
    - less "regular" than relational DB data.
    - cannot (in general) use relational DB tools (e.g. SQL) directly on semistructured data web pages.
    - using NLP tools is overkill

  • In class (and in MP2) we focus on information retrieval from free text documents, which is unstructured data
    - unstructured because no known concise grammar describes English completely.

  • much of the data on the web worth querying is semistructured
    - e.g. a query on amazon.com produces a table of books, prices, reviews, etc.

    [pic of an Amazon.com retrieval].


    - e.g. my portfolio on www.quote.com

    [pic of my portfolio on quote.com]

    2. Why are techniques for retrieving/querying semistructured data important?

  • Lots of valuable information in this format
    - Some places publish data in this format only (e.g. www.imdb.com's movie listings)
    - Often semistructured (HTML) is our only interface to DBMS

  • Being able to retrieve semistructured data turns the web into traditional database (somewhat)
    - Lots of tools, algorithms available to exploit data in this form.
    - Current efforts to make SQL-like query languages for this purpose.

  • "Intelligent" tools such as agents and smart search engines need techniques to"read" sites.
    Example: intelligent agent searches for "restaurants in LA for under $20 that take VISA cards"
    - agent consults two sites: LaWeekly, and Zagat, for data.
    - agent would need to be able to extract the semistructured data into a known format.

    [pic of semistructured data in zagat and laweekly]

    3. Current approaches to retrieving semistrucured data: Wrappers

  • Wrapper: a piece of code attached to a data source which implements a set of extraction rules [Knoblock]
    - extraction rules say where attributes begin and end and where tuples begin and end.

    Example: here's our sample web page again: 

    <html>
<title>Some Country Codes </title><body>
<b>Congo </b> <i> 242 </i > <br >
<b>Egypt </b > <i> 20 </i> <br>
<b>Belize </b> <i> 501 </i> <br>
<b>Spain </b> <i> 34 </i> <br>
</body>
</html>
  • The semistructured data in here has the format: {Country, Area_Code}
    - Rule for extracting Country attr: SkipTo(<b>);
    - Rule for extracting Area_Code attr: SkipTo(<i>);

  • For simple sites like the example above, hand-written rules are used.

  • For more complex sites, like an address list, we use induction
    - Given a few examples, an inductive wrapper tries to learn structure
    - Example: the STALKER algorithm. [Knoblock] 

    <html>
<title>Sample</title>
<body> <p>
513 Pico Blvd., <b>Venice</b>, Phone: 1-<b>800</b>-555-1515<p>
4000 Foo Blvd., <b>Studio City</b>, Phone: (818) 500-1570<p>
1243 Query St., <b> LA </b>,Phone: 1-<b>888</b>-578-2293<p>
403 N. PCH., <b> Redondo Beach </b>, Phone: (310) 798-0008<p>
/body> </html>

  • The user marks the location of the attributes, and STALKER tries to figure out a grammer that describes the locations.

  • The grammar is k-disjunctive, and allows wildcards.

    4. Where can I find out more about retrieving semistructured data?

  • W4F (UPenn) allows users to create wrappers on the fly, and produces XML documents as a by-product.
    - see [this site] for a demo.

  • MIX is UCSD's project related to semistructed data.
    - read more about it [at this site]

    home