Is Data Scraping Legal?

Apr 27,2018 / By John Reilly / No Comment

The question of Data Scraping has been a controversial issue for a long time. The year 2017 witnessed several important developments focused on interpretations of the Computer Fraud and Abuse Act (CFAA) in separate cases involving Craigslist and Linkedin.

What is Data Scraping? Also known as Web Scraping, it is the automated method for extracting large amounts of data from a website, often through the use of Bots.
https://en.wikipedia.org/wiki/Data_scraping

One of the weapons used against Data Scraping is the Computer Fraud and Abuse Act (CFAA), a federal cybersecurity law enacted in 1986 as an amendment to existing computer fraud law (18 U.S.C. Sec. 1030) which had been included in the Comprehensive Crime Control Act of 1984. The CFAA prohibits accessing a computer without authorization, or in excess of authorization if any. It is designed to protect private password mainframe computers.
https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act

In Craigslist, Inc v. Instamotor, Inc, Craigslist claimed that Instamotor scraped Craigslist content to create listings on its own service and sent unsolicited emails to Craigslist users for promotional purposes to sell used cars. Craigslist has become very aggressive in pursuing claims against hackers based on breach of contract (terms of service TOS), violation of the CFAA, and the CAN-SPAM Act (disguised emails).

The case settled in favor of Craigslist for $31 million in a Stipulated Judgment and Permanent Injunction Aug. 3, 2017.
https://bit.ly/2HwzmJF

The next case involved a start up company named hiQ, a member of Linkedin (and subject to Linkedin’s TOS). What hiQ did was to scrape information from Linkedin user profiles and use the scraped data to create workforce data products that it sells to employers. Actually, hiQ tracks user generated changes to profiles in areas like work history and skills, and then uses the data to offer two products, one that helps companies identify employees who are at risk of being recruited away, and another product that helps companies map the skillsets of their employees.

The California U.S. District Court held that hiQ can use web scapers to collect information from PUBLIC Linkedin data. The Key factor in the Linkedin case was that hiQ could access and scrape only public data that was not protected by any authorization technique (such as password protected). Interestingly, the Court granted a preliminary injunction to prohibit Linkedin from employing electronic blocking techniques designed to prevent hiQ from scraping information from public linkedin profiles.
https://bit.ly/2KiDbEi

Linkedin has appealed to the 9^th Circuit Court of Appeals. So we need to wait and see whether data scraping of public data is legal. Still, there is a change in the legal landscape with respect to Data Scraping. Website owners will need to examine how they control or limit access to content they collect from users.

Source: Webinar from https://www.ftcguardian.com/ – I am a charter member

Competition in Residential Real Estate

Apr 21,2018 / By John Reilly / No Comment

Earlier this month the Information Technology and Innovation Foundation (ITIF) organized a panel discussion about the challenges and opportunities to increase competition in the real estate industry through technological innovation.

Participating in the panel were lawyers from NAR, the FTC, and the DOJ, as well as Brian Larson counsel for CMLS. The only non-lawyer in the group was Ben Clark, former broker for eRealty and noted data expert in the real estate industry. It is interesting to hear the broker’s perspective, so we will highlight some of those discussions.

Video of the 90 minute session is available here.

One of the key points from Ben Clark was the need for MLSs to remove any barriers to MLS participants and subscribers receiving access to relevant MLS listing data. This is based on the fact that real estate professionals have the following fiduciary duty under the NAR Code of Ethics Standard of Practice 11-1, “When REALTORS® prepare opinions of real property value or price they must

be knowledgeable about the type of property being valued;
have access to the information and resources necessary to formulate an accurate opinion, and
be familiar with the area where the subject property is located UNLESS lack of any of these is disclosed to the party requesting the opinion in advance.”

You can find some of Ben Clark’s remarks at the following times of the video:

5:55 to 8:08;
35:40 to 39:50;
46:30 to 49:42;
1:03:40 to 1:06:50.
There is also a discussion on MLS Copyright at 1:11:25 to 1:14:05, with a supplemental blog post by Mitch Skinner Click Here

Finally, for an excellent summary of the history of MLS from Brian Larson, go to the video at 27:00 to 32:30

This November marks the expiry date of the Consent Decree in the NAR/DOJ lawsuit 10 years ago. Mark your calendar for June 5, 2018 when the FTC and DOJ will hold a relevant joint Workshop to discuss “What’s New in Residential Real Estate Brokerage Competition” – Click Here

Dot Brands hit the market

Dec 20,2016 / By John Reilly / No Comment

Several recent articles have focused on the increase use of the new Top-level Domains by brands, referred to as Dot Brands.

A multi-million dollar brand name that should be noted is the Dot REALTOR Top-Level Domain. Last month NAR made the switch to www.nar.realtor away from www.realtor.org Check it out.

Tony Kirsch, head of professional services at Neustar, in an article discussing the progress of Dot Brands, “2016: The Year That Was in .brands TLDs”, wrote:

“While some astonishing progress has been made in 2016, the .brand movement is by no means complete. I personally am thrilled to be heading into 2017 with such momentum behind us and I’m optimistic about the continued growth of the .brands space in the New Year.

Already this year we’ve been surprised by the creative approaches many brands have taken and we’re looking forward to seeing which companies will use 2017 to take .brands to places we haven’t considered before.

Perhaps 2017 will be the year in which mainstream advertising strategies adopt .brands as a unique, memorable call-to-action and the marketers of the world find significant value in creating more direct, meaningful customer relationships through smarter digital marketing.

With almost all .brands now added to the Internet, the game is on. Which brands will emerge as frontrunners and truly revolutionize the way we look at digital?

I don’t know the answer, but I sure am excited to find out.”

Some major launches noted were:

Google – consolidating news and updates from over 19 of Google’s corporate and product blogs, ‘The Keyword’ was launched at www.blog.google in October, joining the internet giant’s retail domain name sales play at www.domains.google

Canon – one of a handful of ‘full transitions’ to .brands, the company’s global site is now at www.global.canon , with the existing canon.com domain redirecting to this awesome new site.

As well as: Dell, Deloitte, Chanel, Microsoft, Audi, Cisco, Fage, Orange, Komatsu, Shell, Mini, Lidl, Linde, Bradesco, Hotmail, NTT, UOL, Pictet, Windows, Bloomberg, Sky, Fox, Emerck, Weir and BNP Paribas are just a handful of the other organizations with live .brand websites. http://www.circleid.com/posts/20161219_2016_the_year_that_was_in_brands_tlds/

According to another article “Brand Top-Level Domain and Brand Value” by Guiillaume Pahud,

“Brands are an asset and have a value for organizations, as they generate revenue: Customers are happy to pay a premium for a brand they love. They will show a preference and be loyal to a brand they trust.

Some brands have decided, in 2012, that their name should actually also become the global name for all of the digital assets, and registered their brand name as a Top-Level Domain. Google, for instance, located their new blog platform, where they post news and information, on the domain blog.google. That is a typical example of a branded top-level domain.”

http://www.circleid.com/posts/20161111_brand_top_level_domain_and_brand_value/

Realtors can acquire their own Dot Realtor TLD by going online to www.claim.realtor — the first year is free. If ordering multiple domains or years, use the word discount as the promo code for a 10% saving.

Bots Machine Learning Sales and Selling

Is A Bot In Your Future?

Nov 30,2016 / By John Reilly / No Comment

Is a Bot in Your Future?

For sure in 2017, real estate professionals will be hearing more about the use of Bots, Artificial Intelligence (AI), and machine learning as productivity tools in their practice. Already we are seeing more and more articles and online discussions about Bots.

To give you an idea of what’s to come in this exciting space, here are some highlights from several articles about a San Diego company led by real estate veteran Grant Gould that is making news with the anticipated January 2017 launch of its Automabots service – www.Automabots.com

Real estate professionals, REALTOR® Associations and MLSs will want to take note that initial use of Automabots will be directed at the real estate industry. As mentioned below, AI manifestations can be powerful tools for business efficiency, so now is the time to get informed and use them to your advantage.

Inman’s Real Estate Technology Expert, Craig Rowe, recently reviewed Automabots, –

http://www.inman.com/2016/10/26/automabots-warms-up-buyer-and-seller-leads-til-theyre-hot/

Craig noted that Grant Gould is developing several bots with different purposes in the agent realm one at a time, including BuyerAgentBot, SellerAgentBot (in beta) along with a CRMBot, DigitalMarketingBot and TransactionBot that will eventually form a real estate suite.

During the initial interaction, users indicate their preferred ZIP codes, price, number of beds and baths, and communities they’re interested in to populate a relevant list of homes, which are pulled directly from the MLS, current to the last hour.

After users finish looking at the homes provided, the Automabot asks how often they would like to receive updates on available listings. Gould notes that users’ response to this question serves as a lead qualifier — someone who wants an update every two hours (or daily) is probably more motivated than the casual monthly browser. Today, that entire lead cultivation process is being replaced by text apps and bots. But agents shouldn’t see bots as a threat — the element of data control has been slipping for some time from the industry’s hands, and Kubrick’s eerie vision (of HAL-2000) isn’t quite so dark in real life. Rather, these AI manifestations can be powerful tools for business efficiency, so get informed and use them to your advantage.

Adrian Bishop, Editor for OPP.Today authored a Business Technology Editorial entitled “How Bots Can Turn Leads Into Clients And Boost Sales”
– http://www.opp.today/how-bots-can-turn-leads-into-clients-and-boost-sales/ – In his editorial, based on an interview with Automabots founder Grant Gould, Mr. Bishop wrote:

Essentially, bots are a small application that live in the cloud and utilize the machine learning technology recently made available by the tech giants. These little programs, now referred to as bots, can be taught to perform nearly any digital task that normally would be done by a human being. People can communicate with bots via chat platforms, like; Facebook Messenger, Alexa and the dozens of others.

“We teach real estate bots to do the work, so you can get more done in less time at a lower cost, “says Automabots.

Using natural language technology, your personal bot interacts with prospects to determine their exact real estate needs and delivers the information they desire without having to log in or download an app.

When Grant Gould was asked how does his service work, he responded: “Similar to Siri, Amazon Alexa, and IMB’s Watson, Automabots uses machine learning that is a combination of rule-based and natural language parsing.

“We are teaching the Bot real estate lingo, terminology and the nuances of buying and selling properties. The Lead Capture Bot can engage online users through chat applications, like Facebook Messenger, to assess the consumer needs and then search and display the information right in the chat session. MLS properties, market stats, and home values are just some of the info it can deliver.

“It then asks the user if they would like to get alerts and updates on this information, which results in capturing a lead. As it continues to learn what the user likes and dislikes, it gets smart in finding matching homes.”

Bots save both time and money and boost sales, says the company. “Get a highly skilled Lead Qualifier Bot for just 14 cents an hour. No benefits, sick days, or employer cost. You get a ready-to-work real estate Bot that is specially trained to do a specific job and do it great. And these workers get smarter and more efficient over time.

“Real estate bots can be taught to perform many tasks currently done by humans. We have trained our Bots to greet every person immediately, qualify their buyer and seller needs, and deliver the information they want without using any human resources.

“Having a dependable Lead Qualifier Bot on duty 24/7 to greet buyers and sellers on your website, social profiles and other lead sources will improve your lead conversion rate by as much as 300%. Add a great lead nurturing system and watch business soar.”

In an article in VisionAR entitled “Productivity AI Bots for Real Estate Agents” –
http://visionar.today/index.php/2016/11/22/automabots-makes-productivity-bots-real-estate-agents/ – the author Lauren Twardy interviewed Grant Gould and stated:

“Though Gould expects the Bots’ utility will extend into other industries before long, Automabots is currently focused solely on real estate. Gould is a real estate veteran and understands the kind of help agents need to be productive. Gould likes to call his line of product “Productivity Bots,” which are designed to enhance rather than take over business. He explains, “consumers today want instant gratification,” and his Bots help Realtors provide that gratification by remaining on active duty 24/7. While Realtors can get thousands of visitors to their websites, social media profiles, and other lead sources, the Bots are trained to convert visitors into leads.

Gould’s “Productivity Bots” are productive. The Lead Qualifier Bot is designed to save real estate agents time and money; for just 14 cents an hour, “you get a ready-to-work real estate Bot that is trained to do a specific job and do it great.” The Automabots website explains the basic functions of the Bot:

“Greets Every Prospect.” It is designed to engage potential customers 24/7 on real estate websites, social media, and other listing sites. This provides instant gratification for consumers, which they don’t receive with website contact forms or email.
“Qualifies Buyer and Seller Needs.” The Bot can interact with customers to determine what they’re looking for and can dole out information from the massive amount of stored data.
“Captures Leads and Intel.” Not only can it record information about prospects, but the Bot can also transfer information directly to the Realtor or to an automatic lead nurturing program.
Of course, the functions of this AI are meant to grow beyond these basics. Automabots “build intelligent Bots that [they] teach to perform important tasks and that gets smarter as it does its job.”

2017 could very well be the year of the Bot in real estate. Stay tuned and explore ways the services of a Bot can help you be more efficient and productive.

Uncategorized

Cognition, Nuance, and Data Points…

Oct 16,2016 / By Saul Klein / No Comment

Don’t let anyone kid you, they don’t know the value of your data. Don’t give it away before you know what it is worth.

“A new IBM global study of cognitive early adopters reveals nearly six in ten view cognitive computing as a “must have” for organizations to remain competitive within the next few years.

Digital businesses have been disrupting virtually every industry in the past several years, fundamentally transforming how they engage with customers and markets. Data is both the fuel and the product of digital transformation. By 2020, about 1.7 megabytes of information will be created every second for every person on Earth. While digital data represents a vast opportunity, its velocity, diversity, and complexity presents a major challenge. In fact, an estimated 99.5 percent of that digital data is never analyzed.

To keep a competitive edge, organizations will need to maximize the insights — the “digital intelligence” — they create from the vast data flows being generated by the digital revolution.

http://bit.ly/2eljBHE

Domain Names Dot REALTOR

Dot WEB (.WEB) sold at auction for $135 million

Jul 31,2016 / By John Reilly / No Comment

Dot WEB (.WEB) sold at auction for $135 million on July 29

First, it was the new Top-level Domain (TLD) .APP that was acquired at auction for a $25 million bid by Google; recently it was .SHOP that was won with a $41.5 million bid from a Japanese company.

And now, the contentious battle among 7 companies who applied to operate the .WEB suffix has ended in a winning bid of $135 million, with speculation that the winning bid was backed by .COM operator Verisign. For that kind of money and contention, the domain market indicates that .WEB could be even bigger than .COM

So, who gets the $135 million? It goes to ICANN, who has now banked over $235 million as a result of ICANN’s last resort auction mechanism that is used when several parties applied for the same suffix but were unable to resolve this contention by themselves.

And what will ICANN do with all this auction money? Good question! See http://bit.ly/2apFOlM

April 1

First Quarter Report – April 1, 2016

Apr 1,2016 / By Saul Klein / No Comment

Once again the first quarter of the year has passed, and as we have done for the past 10 years or so, John and I will summarize events of note, April1, 2016. This is an election year and we have breaking political news, as well as THE major stories in the real estate industry, which is where we will start…

The Perfect Storm:

In a blow to the real estate industry, President Obama issued an Executive Order today, effective immediately, eliminating the employment status of independent contractor in a number of industries, including the real estate industry. This represents a paradigm shift to the current model of real estate sales. All real estate licensees will become employees of their broker, have the right to organize and collective bargain, receive health care coverage, retirement plans, and be paid minimum wage.

In another major blow to the Real Estate Industry, the U.S. Department of Justice, today filed a complaint in federal court, naming every MLS in the U.S. as Respondents. The complaint alleges, among other things, actions involving conspiracy, accusing the MLSs of per se violations of the Sherman Anti Trust Act, as facilitators of illegal collaboration between competitors. “New online technologies, companies, and innovation make the structure and purpose of MLS obsolete” stated a DOJ representative, who agreed to speak with us with assurances of anonymity.

And for the Hat Trick, Google announced that it is returning to real estate in a big way, opening its “National Listing Service” to the public, and purchasing Move, Inc (the operator of REALTOR.com) from Rupert Murdock.

newspaper_article

But that’s not all, keep reading!

If you ever wondered what Google planned to do with all of that “white space” on the search page, now you know, they are offering a free listing service, allowing real estate offered for sale to be advertised on the Google Real Estate Site, for free, launched off of the Goggle Search Page. Consumers are waking up to Google Real Estate…It will offer disrupting tools and applications to change the way real estate is bought and sold, virtually eliminating fees to buyers and sellers. This will redefine the role of the real estate professional, and fill the gap should the DOJ take out MLSs, as mentioned in an earlier story.

google_home_page-001

And, the world of politics leaves nothing to the imagination.

Our inside source has informed us that at 3:00 PM, EDT, Donald Trump will end his campaign and quest for the presidency of the United States. It is rumored, amongst those in the know, that Trump and Ted Cruz have struck a deal. In return for Trump resigning from the race, Ted Cruz, if elected, will build that wall Trump has been promising, along the border with Mexico. Ted Cruz, in turn, promised Donald Trump that he would deliver him the contract to build the wall, and Trump will be allowed to put the Trump brand on the wall, which will be able to be viewed from space. Masterful positioning, negotiating and branding by The Donald.

Also at 3:00 PM, EDT, we are told, the Department of Justice will issue indictments against Hillary Clinton, 27 different Causes of Action. Hillary could not be reached, but President Bill Clinton responded: “What difference does it make? She is the only one of the four remaining candidates prepared for the job of President. Are you going to let trust get in the way?”

Bernie Sanders was asked for his comments, but he could not be reached, taking the week off to celebrate Purim with his family.

And lastly, Jesse Ventura, and Arnold Schwarzenegger announced that they are running as President and Vice President respectively on the Peace and Freedom Party ticket. When asked why they decided to run, they responded that this is the year for outsiders. Hulk Hogan will serve as Secretary of Defense.

It is an amazing world and an amazing time, and once again we are happy to bring you the news, April 1, 2016.

Domain Names Dot REALTOR Uncategorized

Universal Acceptance — TLDs

Feb 29,2016 / By John Reilly / No Comment

With over 12 million new Top-level Domains in operation, it appears that a few software developers and website owners have yet to update their systems to keep pace with this evolving domain name system. Check out this helpful link created by the Universal Acceptance Steering Group —http://www.universalacceptance.link/

The Universal Acceptance Steering Group is a community-based

team working to share their vision for the Internet of the future with those who construct this space: coders. The group’s primary objective is to help software developers and website owners understand how to update their systems in light of the rapid acceptance of the new TLDs. Its primary message is that Universal Acceptance will enable the next billion users to build their own spaces and identities online.

Of special interest to developers and programmers is the Domain Test https://github.com/google/domaintest

Domain Test is a tool designed to help developers test their applications for compatibility with new top-level domains (TLDs). Developed by Google and launched in a partnership between Google Registry and Ausregistry, CentralNic, Donuts, RightSide, and Uniregistry, Domain Test is an open source project available under the Apache 2 license and can be used across 145 new TLDs. It is freely available for use and modification.

This repository contains the documentation and code for Domain Test. For clarity, the documentation uses the term “new TLDs” to refer to the universe of new generic top-level domains (gTLDs), new country-code top-level domains (ccTLDs), and internationalized domain names (IDNs).

The Domain Test service runs on AppEngine and is available for any developer to use. However, depending on what type of new TLD you want to test, you can substitute any of the strings in the Domain Test TLDs section of this documentation.

Of interest, the Universal Acceptance site also references a number of examples of new TLDs currently in use — Top Websites using New gTLDs – http://list.ly/list/Rg5-top-websites-using-new-gtlds

Shopping on .SHOP

Feb 15,2016 / By John Reilly / No Comment

In February of 2015, the Internet community was shocked to find out that someone paid $25,001,000 in an auction to acquire the new Top-level Domain (TLD) for .APP (Dot APP). That someone, of course, was none other than Google. And just last month, on January 27, 2016, the new TLD for .SHOP (Dot Shop), was sold at auction for a whopping $41,501,000. The winning bidder was GMO Domain International, Inc from Tokyo. They outlasted Amazon and Google in a bidding war that went 14 rounds, the final round going from $36,800,000 to $46,000,000.
https://gtldresult.icann.org/application-result/applicationstatus/auctionresults

We are talking some serious money here in the ecommerce space, a space in which there are now over 12 million new TLDs.

How do we even get to the auction stage? First, for anyone to apply to operate a new TLD, there is a $185,000 application fee. In the event more than one applicant applies for the same domain (called a ‘string’), those competing entities move to a string contention status under ICANN rules. ICANN is a not-for-profit public-benefit corporation with participants from all over the world dedicated to keeping the Internet secure, stable and interoperable.

According to ICANN, “Auctions are the mechanism of last resort to resolve string contention within the New gTLD Program. ICANN expects that most string contention will be resolved through other means before reaching an Auction conducted by ICANN’s authorized Auction service provider, Power Auctions LLC. However, there is a possibility that significant funding will accrue as a result of several Auctions. Auction proceeds will be reserved and earmarked until the Board determines a plan for the appropriate use of the funds through consultation with the community. Auction proceeds are net of any Auction costs. Auction costs may include initial set-up costs, auction management fees, and escrow fees.”

212 of the 233 contention sets are now resolved, the majority having self-resolved, but a few were resolved via ICANN’s auction process (“the method of last resort”). In some cases, contending applicants formed joint ventures in their efforts to resolve string contention. In other cases, some applicants withdrew after receiving some compensation from the surviving applicant — hopefully in an amount to at least compensate the withdrawing party for their $185,000 initial application fee.

In one case of a contention over the new TLD .coupon (Dot coupon), the battle was between Donuts, Inc and the Coupon.com company. Donuts prevailed. According to an SEC filing, Coupons.com received $4.8 million dollars, apparently based on a privately negotiated agreement. As one pundit noted, it only cost Coupons.com $185,000 to file its application for .Coupons TLD, making this the best coupon the company ever redeemed.

And what about the $41,501,000 winning bid for .SHOP – who got that money? Well, ICANN of course. In fact, in the 15 auctions that have concluded since June 2014, ICANN has amassed $101,357,812 in net proceeds. Some of the top auction winning bids besides Google’s .APP, were .TECH for $6,706,000, .REALTY for $5,588,888, and .MLS for $3,359,000 by CREA. https://newgtlds.icann.org/en/applicants/auctions/proceeds

According to ICANN, “Possible uses of auction funds include formation of a foundation with a clear mission and a transparent way to allocate funds to projects that are of interest to the greater Internet community, such as grants to support new gTLD applications or registry operators from communities in subsequent gTLD rounds, the creation of an ICANN-administered/community-based fund for specific projects for the benefit of the Internet community, the creation of a registry continuity fund for the protection of registrants (ensuring that funds would be in place to support the operation of a gTLD registry until a successor could be found), or establishment of a security fund to expand use of secure protocols, conduct research, and support standards development organizations in accordance with ICANN’s security and stability mission.”

With more than 1500 TLD applications submitted to ICANN in the first round, there are sure to be companies that do not turn their TLDs into successful businesses. But with such a large amount of investment money at stake, we can expect to see a lot of ecommerce activity built around many of the new TLD ventures.

Uncategorized

What would it be worth to be able to predict the future…with accuracy?

Feb 14,2016 / By Saul Klein / No Comment

This is about the data, not the politics. Why all of the fuss over data? With data, you can accurately predict the future. Being able to tell the future could be a great, competitive advantage.

Don’t be so quick to give your MLS data away to outsiders.

Google calls it in New Hampshire.