| dbo:abstract |
Common Crawl is a nonprofit 501(c)(3) organization that crawls the web and freely provides its archives and datasets to the public. Common Crawl's web archive consists of petabytes of data collected since 2011. It completes crawls generally every month. Common Crawl was founded by Gil Elbaz. Advisors to the non-profit include Peter Norvig and Joi Ito. The organization's crawlers respect nofollow and robots.txt policies. Open source code for processing Common Crawl's data set is publicly available. The Common Crawl dataset includes copyrighted work and is distributed from the US under fair use claims. Researchers in other countries have made use of techniques such as shuffling sentences or referencing the common crawl dataset to work around copyright law in other legal jurisdictions. (en) Common Crawl (literalmente rastreo común) es una organización sin fines de lucro 501 (c) (3) que rastrea la web y proporciona libremente sus archivos y conjuntos de datos al público. El archivo web de Common Crawl consta de petabytes de datos recopilados desde 2008. Completa el rastreo en general una vez al mes. Common Crawl fue fundada por Gil Elbaz. También están Peter Norvig y Joi Ito como asesores de la organización sin fines. Sus rastreadores (crawlers) respetan las políticas nofollow y robots.txt. El código fuente usado para procesar el conjunto de datos de Common Crawl es abierto y se encuentra disponible públicamente. (es) Common Crawl är en ideell organisation som genomsöker webben och fritt tillhandahåller sina arkiv och datamängder till allmänheten. Common Crawls webbarkiv består av petabyte data som samlats in sedan 2011. Den genomför genomsökningar i allmänhet varje månad. (sv) |
| dbo:foundedBy |
dbr:Gil_Elbaz |
| dbo:keyPerson |
dbr:Carl_Malamud dbr:Peter_Norvig dbr:Kurt_Bollacker dbr:Nova_Spivack |
| dbo:language |
dbr:English_language |
| dbo:location |
dbr:Los_Angeles,_California dbr:San_Francisco,_California |
| dbo:type |
dbr:501(c)(3) |
| dbo:wikiPageExternalLink |
http://commoncrawl.org/ https://commoncrawl.org/connect/blog/ https://github.com/commoncrawl/ https://groups.google.com/forum/%3Ffromgroups%23!forum/common-crawl |
| dbo:wikiPageID |
40739436 (xsd:integer) |
| dbo:wikiPageLength |
12079 (xsd:nonNegativeInteger) |
| dbo:wikiPageRevisionID |
1123122833 (xsd:integer) |
| dbo:wikiPageWikiLink |
dbr:Carl_Malamud dbr:Benelux dbc:Web_archiving dbc:Web_archiving_initiatives dbr:Joi_Ito dbr:Peter_Norvig dbr:Kurt_Bollacker dbr:English_language dbr:Los_Angeles,_California dbr:501(c)_organization dbr:Timnit_Gebru dbr:Web_crawler dbr:GPT-3 dbr:Jurisdiction dbr:501(c)(3) dbr:Amazon_Web_Services dbr:Fair_use dbr:Nofollow dbr:Nonprofit_organization dbr:Nova_Spivack dbr:ARC_(file_format) dbc:Internet-related_organizations dbr:Blekko dbr:Web_ARChive dbr:Apache_Software_Foundation dbr:Metadata dbr:Nutch dbr:Search_engine_optimization dbr:Gil_Elbaz dbr:SURFsara dbr:Web_archiving dbr:San_Francisco,_California dbr:Robot_exclusion_standard |
| dbp:companyType |
501 (xsd:integer) |
| dbp:founder |
dbr:Gil_Elbaz |
| dbp:keyPeople |
dbr:Carl_Malamud dbr:Joi_Ito dbr:Peter_Norvig dbr:Kurt_Bollacker dbr:Nova_Spivack |
| dbp:language |
dbr:English_language |
| dbp:location |
San Francisco, California; Los Angeles, California, United States (en) |
| dbp:name |
Common Crawl (en) |
| dbp:wikiPageUsesTemplate |
dbt:Infobox_dot-com_company dbt:Reflist dbt:Short_description dbt:Url |
| dct:subject |
dbc:Web_archiving dbc:Web_archiving_initiatives dbc:Internet-related_organizations |
| gold:hypernym |
dbr:Organization |
| rdf:type |
owl:Thing dbo:Company schema:Organization dul:Agent dul:SocialPerson dbo:Agent wikidata:Q24229398 wikidata:Q43229 wikidata:Q4830453 yago:Abstraction100002137 yago:Company108058098 yago:Group100031264 yago:Institution108053576 yago:Organization108008335 yago:WikicatInternetCompanies yago:YagoLegalActor yago:YagoLegalActorGeo yago:YagoPermanentlyLocatedEntity dbo:Organisation yago:SocialGroup107950920 |
| rdfs:comment |
Common Crawl är en ideell organisation som genomsöker webben och fritt tillhandahåller sina arkiv och datamängder till allmänheten. Common Crawls webbarkiv består av petabyte data som samlats in sedan 2011. Den genomför genomsökningar i allmänhet varje månad. (sv) Common Crawl is a nonprofit 501(c)(3) organization that crawls the web and freely provides its archives and datasets to the public. Common Crawl's web archive consists of petabytes of data collected since 2011. It completes crawls generally every month. Common Crawl was founded by Gil Elbaz. Advisors to the non-profit include Peter Norvig and Joi Ito. The organization's crawlers respect nofollow and robots.txt policies. Open source code for processing Common Crawl's data set is publicly available. (en) Common Crawl (literalmente rastreo común) es una organización sin fines de lucro 501 (c) (3) que rastrea la web y proporciona libremente sus archivos y conjuntos de datos al público. El archivo web de Common Crawl consta de petabytes de datos recopilados desde 2008. Completa el rastreo en general una vez al mes. (es) |
| rdfs:label |
Common Crawl (en) Common Crawl (es) コモン・クロール (ja) Common Crawl (sv) |
| owl:sameAs |
freebase:Common Crawl yago-res:Common Crawl wikidata:Common Crawl dbpedia-es:Common Crawl dbpedia-ja:Common Crawl dbpedia-sv:Common Crawl dbpedia-tr:Common Crawl https://global.dbpedia.org/id/FsQu |
| prov:wasDerivedFrom |
wikipedia-en:Common_Crawl?oldid=1123122833&ns=0 |
| foaf:isPrimaryTopicOf |
wikipedia-en:Common_Crawl |
| foaf:name |
Common Crawl (en) |
| is dbo:wikiPageRedirects of |
dbr:Commoncrawl.org dbr:CommonCrawl |
| is dbo:wikiPageWikiLink of |
dbr:Kurt_Bollacker dbr:LAION dbr:Commoncrawl.org dbr:Apache_Nutch dbr:StormCrawler dbr:Wayback_Machine dbr:GPT-3 dbr:Nova_Spivack dbr:Global_Multimedia_Protocols_Group dbr:James_Hendler dbr:Transformer_(machine_learning_model) dbr:Stable_Diffusion dbr:Gil_Elbaz dbr:Web_archiving dbr:CommonCrawl |
| is foaf:primaryTopic of |
wikipedia-en:Common_Crawl |