Quantcast

Got Homework?

Connect with other students for help. It's a free community.

  • across
    MIT Grad Student
    Online now
  • laura*
    Helped 1,000 students
    Online now
  • Hero
    College Math Guru
    Online now

Here's the question you clicked on:

55 members online
  • 0 replying
  • 0 viewing

sasogeek Group Title

how can you tell what a website is about from its source code?

  • 2 years ago
  • 2 years ago

  • This Question is Closed
  1. sasogeek Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    ok so i've written a crawler and i'm storing the links in a database but i don't want all links in one database. i want specific links in specific databases. so that links who's pages are about specific topics or subjects or categories of some ideas are placed in specific databases. and to do that i need to know what the page that link is from is about.

    • 2 years ago
  2. sasogeek Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    or even if same database, different tables, i'd still need to find a way to place them in the right places... :/

    • 2 years ago
  3. across Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    You want something that can 'read' the links, 'know what they're about', and store them accordingly?

    • 2 years ago
  4. sasogeek Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    yes :)

    • 2 years ago
  5. sasogeek Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    for now, i can do the reading part, "know what they're about" is what i'm in search of atm...

    • 2 years ago
  6. across Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    This is artificial intelligence, specifically, natural language processing. You want something that will go through a website and store it in a finite number of categories. Either that, or you can find a gigantic table of website-category associations.

    • 2 years ago
  7. across Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    It has to be customized, however, since the word 'category' is rather vague and you may have it defined it differently than someone else.

    • 2 years ago
  8. sasogeek Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    a finite number of categories is what i plan to start with, but as time goes on, users are allowed to submit categories of their interest that are not in the default list and that might be added later depending on how many people really would need such a category

    • 2 years ago
  9. across Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    This is exactly what's going on at math.stackexchange.com, except that the categorization is performed by users, not parsers. This is due to the immense difficulty of processing natural language. ;)

    • 2 years ago
  10. sasogeek Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    the idea is certainly to make it customizable so that users get data based on personal preferences... but i guess everything starts from somewhere, that's where i'm looking for right now :/

    • 2 years ago
  11. sasogeek Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    are meta tags a good idea?

    • 2 years ago
  12. across Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    It depends on the 'context.'

    • 2 years ago
  13. across Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    The biggest lesson that I learned as a CS undergrad is that the field has to be insanely well-defined for it to be feasible.

    • 2 years ago
  14. sasogeek Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    what would you suggest i attempt?

    • 2 years ago
  15. across Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    Find a Ph.D., or group thereof, carrying out research in the field of natural language processing and assist them in their pursuit of the betterment of e-mankind!

    • 2 years ago
  16. across Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    http://cms.dt.uh.edu/faculty/chenp/

    • 2 years ago
  17. across Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    Notice the guy's working on a "node-based web crawler."

    • 2 years ago
  18. sasogeek Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    how am i supposed to help? :s

    • 2 years ago
  19. sasogeek Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    assist i suppose is the right word in place of "help" :/ but still ....

    • 2 years ago
  20. across Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    You're now in a university. ;) Let's talk to some professors!

    • 2 years ago
  21. sasogeek Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    well i'm guessing it's not possible that i get in contact by any chance with the person in the link you gave me with respect to his research.... :/

    • 2 years ago
  22. across Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    He's just one in a million Ph.D.s carrying out research in this field.

    • 2 years ago
  23. sasogeek Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    i see, well it seems difficult than it looks anyway. :/ looks like i have a long way ahead of me lol

    • 2 years ago
  24. GMoore Group Title
    Best Response
    You've already chosen the best response.
    Medals 1

    yo...you are thinking to much into what you are trying to progress forward on...lol... keep it simple.. and build parts and other fixs... i have a book that you might find interest in..it's called Constructing intelligent Agents with Java.. read the book from start to finish then step back , make a plan then go with it..

    • 2 years ago
  25. Pradius Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    ehhh what about metatags? they should tell you what are the links about

    • 2 years ago
  26. ktobah Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    Yes, the meta information can be helpful. Also if you try to get the content of the link then do some indexing for example if you know which word repeated more that can help you to know the context of this link (document). I know it's heavy, because you need to process many links so a lot of content, but it's just an idea!

    • 2 years ago
  27. sasogeek Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    Thanks guys :) for now I'll be using the meta tags but in later development, i may change it to something sophisticated, i just want it working okay for now :) thanks for mentioning it again x

    • 2 years ago
  28. ktobah Group Title
    Best Response
    You've already chosen the best response.
    Medals 0

    Welcome

    • 2 years ago
    • Attachments:

See more questions >>>

Your question is ready. Sign up for free to start getting answers.

spraguer (Moderator)
5 → View Detailed Profile

is replying to Can someone tell me what button the professor is hitting...

23

  • Teamwork 19 Teammate
  • Problem Solving 19 Hero
  • You have blocked this person.
  • ✔ You're a fan Checking fan status...

Thanks for being so helpful in mathematics. If you are getting quality help, make sure you spread the word about OpenStudy.

This is the testimonial you wrote.
You haven't written a testimonial for Owlfred.