First question, do you have an index on the table.

That should help.

A non-clustered index on place would help, as SQL Server then only
would have to scan that index, and not the entire difference. But it
seems that the table has two columns. In such case, the index would not
reduce execution time that much.

A better alternative may be to define an indexed view that maintains
the count:

  CREATE TABLE pagehits (pageid varchar(20) NOT NULL,
                         viewtime datetime NOT NULL,
                         PRIMARY KEY (pageid, viewtime)
  )
  go
  CREATE VIEW pagecount WITH SCHEMABINDING AS
     SELECT pageid, cnt = COUNT_BIG(*)
     FROM   dbo.pagehits
     GROUP  BY pageid
  go
  CREATE UNIQUE CLUSTERED INDEX pagecount_ix ON pagecount(pageid)
  go
  SELECT TOP 10 pageid, cnt
  FROM   pagecount WITH (NOEXPAND)
  ORDER  BY cnt
  go
  DROP VIEW pagecount
  DROP TABLE pagehits

I added the NOEXPAND hint to the query, since it's only on Enterprise
Edition, the optimizer considers indexed views.

Note that this could have effect on performance when writing to the table.

Fastest returning query would be on a separate table that maintains the
count to the granularity you would require in your output (week, day, or
hour). A query that requires more than hourly granularity can still be done
on the source table.

For each new week|day|hour add a row for each page to be tracked. This can
be done dynamically during monitoring but it is better to do it ahead of
time.

Then put an insert trigger on your million row table that will increment the
count on the proper row of the table.

An alternative would be a temp table that contains a subset copy of the
source data for the time range most likely to queried. For example a table
that contains the last 31 days of data, the 32nd day of data
deleted/archived every night.